Spacy Transformers Versions Save

🌙 This release is a pre-release and requires spaCy v3 (nightly).

✨ New features and improvements

• Rewrite library from scratch for spaCy v3.0.
• Transformer component for easy pipeline integration.
• TransformerListener to share transformer weights between components.
• Built-in registered functions that are available in spaCy if spacy-transformers is installed in the same environment.

📖 Documentation

• Embeddings, Transformers and Transfer Learning: How to use transformers in spaCy
• Training Pipelines and Models: Train and update components on your own data and integrate custom models
• Layers and Model Architectures: Power spaCy components with custom neural networks
• Transformer: Pipeline component API reference
• Transformer architectures: Architectures and registered functions

• Fix issue #204: Store model_type in tok2vec config to fix speed degradation

⚠️ This release requires downloading new models.

• Update spacy-transformers for spaCy v2.3
• Update and extend supported transformers versions to >=2.4.0,<2.9.0
• Use transformers.AutoConfig to support loading pretrained models from https://huggingface.co/models
• #123: Fix alignment algorithm using pytokenizations

Thanks to @tamuhey for the pull request!

Bug fixes related to alignment and truncation:

• #191: Reset max_len in case of alignment error
• #196: Fix wordpiecer truncation to be per sentence

Enhancement:

• #162: Let nlp.update handle Doc type inputs

Thanks to @ZhuoruLin for the pull requests and helping us debug issues related to batching and truncation!

Update to newer version of transformers.

This library is being rewritten for spaCy v3, in order to improve its flexibility and performance and to make it easier to stay up to date with new transformer models. See here for details: https://github.com/explosion/spacy-transformers/pull/173

Fix various alignment and preprocessing bugs.

• Downgrade version pin of importlib_metadata to prevent conflict.
• Fix issue #92: Fix index error when calculating doc.tensor.

Thanks to @ssavvi for the pull request!

⚠️ This release requires downloading new models. Also note the new model names that specify trf (transformers) instead of pytt (PyTorch transformers).

• Rename package from spacy-pytorch-transformers to spacy-transformers.
• Update to spacy>=2.2.0.
• Upgrade to latest transformers.
• Improve code and repo organization.

• Fix serialization.
• Update pytorch-transformers to support DistilBERT.
• Add pre-packed DistilBERT model.

• Add out-of-the-box support for RoBERTa.
• Add pre-packaged RoBERTa model.
• Update to pytorch-transformers v1.1.
• Fix serialization when model was saved from GPU.