GitLab

* forward contrib credits from suggestion

* add `accelerate` support for BART-like models
Co-authored-by: sgugger <sgugger@users.noreply.github.com>

* Let inputs of fast tokenizers be tuples as well as lists

* Update src/transformers/tokenization_utils_fast.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

* Style
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

* Wip

* Add safetensors support for TensorFlow

* First tests

* Add final test for now

* Retrigger CI like this

* Update src/transformers/modeling_tf_utils.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

* add resources for gpt2

* add pipeline icons and community resources

* add bloom resources

* add pipeline icon

* add resources for t5

* add pipeline icons and community resources

* add roberta resources

* fix typo

* add `accelerate` support for M2M100

* fix device set nit

* added typing to call in TFPegasusModel and TFPegasusForConditionalGeneration

* fixed type for TFPegasusForConditionalGeneration call

* Fix tests when running on GPU

* Fix tests that require mp.set_start_method

* Add pegasus_x

* ViTMSN

* ESM
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Convert None logits processor/stopping criteria to empty list.

* Initialize stopping_criteria, logits_processor in generate.

* Default stopping_criteria, logits_processor to None.
Co-authored-by: Chandler May <chandler.j.may@gmail.com>

* For CLIP

* Others

* update
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* add first generation tutorial

* [Flax] Add subfolder functionality

* [Flax] Add subfolder functionality

* up

* finish

* delete file and re-add test

* add `flan-t5` documentation page

* Update README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* add more content

* revert `_toctree` modif

* revert `toctree` modif - 2

* Update README.md

* Revert "Update README.md"

This reverts commit 56607144299c5fdf7b18abdb776efd0d03287727.

* Update README_es.md

* Update README_zh-hans.md

* Update README_zh-hant.md

* Update README_ko.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Fix test_save_load_fast_init_to_base

* Fix test_save_load_fast_init_to_base

* update
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Change the import of kenlm from github to pypi

* Change the import of kenlm from github to pypi in circleci config

* Fix code quality issues

* Fix isort issue, add kenlm in extras for audio

* Add kenlm to deps

* Add kenlm to deps

* Commit 'make fixup' changes

* Remove version from kenlm deps

* commit make fixup changes

* Remove manual installation of kenlm

* Remove manual installation of kenlm

* Remove manual installation of kenlm

* Add missing information on token_type_ids for roberta model

* Fix code format issues

* Fix code format issues

* Add more explicit document for token_type_ids for roberta

* Fix flake8 issues

* Fix flake8 issues

* Fix flake8 issues

* Fix flake8 issues

* Fix flake8 issues

* no conv bn folding in ipex

* no flag in training

* comment
Co-authored-by: Sander Land <sander@chatdesk.com>

swap "right" and "left" so description is correct.

* Update documentation_tests.txt

* Update configuration_mbart.py

* Update configuration_mctct.py

* Update configuration_layoutlm.py

* Update configuration_layoutlmv2.py

* Update configuration_layoutlmv3.py

* Update documentation_tests.txt

* Factored out some code in the image-segmentation pipeline

Re-enable `small_model_pt`.

Re-enable `small_model_pt`.

Enabling the current test with the current values.

Debugging the values on the CI.

More logs ? Printing doesn't work ?

Using the CI values instead. Seems to be a Pillow sensitivity.

Added a test showcasing that models not supporting some tasks get a
clear error.

Factored out code.

Further factor out.

Fixup.

Bad rebase.

Put `panoptic` before `instance` as it should be a superset.

* Fixing tests.

* Adding subtasks tests

+ Fixes `instance` segmentation which was broken due to default and
non kwargs arguments.

* Fix bad replace.

* Update doc for revision and token

* Update src/transformers/configuration_utils.py
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

* Push changes on other from_pretrained methods
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

* Fix model-tokenizer mapping
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Support for Vilt in v1.9

* Skip if not higher or equal than 1.10

* Move test :)

* I am bad at python

* [ custom_models.mdx ] - Translated to Portuguese the custom models tutorial.

* [ run_scripts.mdx ] - Translated to Portuguese the run scripts tutorial.

* [ converting_tensorflow_models.mdx ] - Translated to Portuguese the converting tensorflow models tutorial.

* [ converting_tensorflow_models.mdx ] - Translated to Portuguese the converting tensorflow models tutorial.

* [ serialization.mdx ] - Translated to Portuguese the serialization tutorial.

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Translated multiple_choice.mdx, question_answering.mdx. Added them to _toctree.yml

* Added translation for a missed line.

* Update _toctree.yml as per Omar's suggestions

* Update multiple_choice.mdx as per Omar's comments

* Updt question_answering.mdx as per Omar's comments

* Refactor conversion function

* Remove dupe line

* Fixes

* Fixes

* Use the right variable...

* Fix last test

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* print first diff line intead of first code part line

* fix style

* Update expected values

* fix style
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>