GitLab

* Add ALBERT to convert command of transformers-cli

* Document ALBERT tf to pytorch model conversion

* docs: fix link to token classification (NER) example

* examples: fix links to NER scripts

* Create README.md

* Update model_cards/lserinol/bert-turkish-question-answering/README.md
Co-authored-by: Julien Chaumond <chaumond@gmail.com>

* Fix the issue to properly run the accumulator with TF 2.2

* Apply style

* Fix training_args_tf for TF 2.2

* Fix the TF training args when only one GPU is available

* Remove the fixed version of TF in setup.py

* adapt convert script

* update convert script

* finish

* fix marian pretrained docs

- MarianSentencepieceTokenizer - > MarianTokenizer
- Start using unk token.
- add docs page
- add better generation params to MarianConfig
- more conversion utilities

* [TPU] Doc, fix xla_spawn.py, only preprocess dataset once

* Update examples/README.md

* [xla_spawn] Add `_mp_fn` to other Trainer scripts

* [TPU] Fix: eval dataloader was None

* example updated to use generation pipeline

* Update model_cards/LorenzoDeMattei/GePpeTto/README.md
Co-authored-by: Julien Chaumond <chaumond@gmail.com>

* Create README.md

* Adding code fence around code block

model card for my De Novo Drug discovery model using MLM

* fix PR

* move tests to correct place

* Add AlbertForPreTraining and TFAlbertForPreTraining models.

* PyTorch conversion

* TensorFlow conversion

* style
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>

* README

* Update README.md

* Created using Colaboratory

* [examples] reorganize files

* remove run_tpu_glue.py as superseded by TPU support in Trainer

* Bugfix: int, not tuple

* move files around

cc @patrickvonplaten @thomwolf

As per https://github.com/huggingface/transformers/pull/3934#discussion_r421307659

* wip

* wip

* a last wip

* Better logging when using TPUs

* Correct argument name

* Tests

* fix

* Metrics in evaluation

* Update src/transformers/training_args.py

* [tpu] Use launcher script instead

* [tpu] lots of tweaks

* Fix formatting
Co-authored-by: Julien Chaumond <chaumond@gmail.com>

Ensure fast tokenizer can construct tensor without pad token if only one sample is provided. (#4201)

* Rewritten batch support in pipelines.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>

* Fix imports sorting 🔧

Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>

* Set pad_to_max_length=True by default on Pipeline.

* Set pad_to_max_length=False for generation pipelines.

Most of generation models doesn't have padding token.

* Address @joeddav review comment: Uniformized *args.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>

* Address @joeddav review comment: Uniformized *args (second).
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>

* fix example reformer

* fix error message and example docstring

* improved error message