GitLab

* All done

* Link to the tutorial

* Typo fixes
Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>

* Add metnion of the return_xxx args
Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>

* update tests for fast tokenizers + fix small bug in saving/loading

* better tests on serialization

* fixing serialization

* comment cleanup

* Use _static from master everywhere

* Copy to existing too

* fix weirdness in roberta/bart for mnli trained checkpoints

* black compliance

* isort code check

* Add dehatebert-mono-arabic readme card

* Update dehatebert-mono-arabic model card

* model cards for Hate-speech-CNERG models

* Cleaning TensorFlow models

Update all classes


stylr

* Don't average loss

* Try with the same command

* Try like this

* fix use cache

* add bart use cache

* fix bart

* finish bart

* add benchmark for all kinds of models

* improved import

* delete bogus files

* make style

* Create README.md

* Update model_cards/a-ware/roberta-large-squad-classification/README.md
Co-authored-by: Julien Chaumond <chaumond@gmail.com>

Fix/add information in README.md

electra_large_discriminator_squad2_512 Question Answering LM

* Fix PABEE division by zero error

* patience=0 by default

* Only put tensors on a device

* Type hint and unpack list comprehension

* Add version control menu

* Constify things
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Apply suggestions from code review
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Julien Chaumond <chaumond@gmail.com>

* Add return lengths

* make pad a bit more flexible so it can be used as collate_fn

* check all kwargs sent to encoding method are known

* fixing kwargs in encodings

* New AddedToken class in python

This class let you specify specifique tokenization behaviors for some special tokens. Used in particular for GPT2 and Roberta, to control how white spaces are stripped around special tokens.

* style and quality

* switched to hugginface tokenizers library for AddedTokens

* up to tokenizer 0.8.0-rc3 - update API to use AddedToken state

* style and quality

* do not raise an error on additional or unused kwargs for tokenize() but only a warning

* transfo-xl pretrained model requires torch

* Update src/transformers/tokenization_utils.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* improve mem handling

* improve mem for pos ax encodings

* [Modelcard] bart-squadv2

* Update README.md

* Update README.md