GitLab

* Make `MobileBert` tokenizers independent from `Bert`

* Update src/transformers/models/mobilebert/tokenization_mobilebert.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Fixed the name in the error message

* Update src/transformers/models/mobilebert/tokenization_mobilebert.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Removed extra space from the "copied" comment
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* update marian default vocab size

* also update docstring

* Fix XCLIP doc tests

* Add model to doc test list

* Fix tests

* Replaced ElectraTokenizerFast with  BertTokenzier class

* Fixed Styling issue
Co-authored-by: vishwaspai <vishwas.pai@emplay.net>

* made tokenization_roformer independent of bert

* added missing imports

* added missing function and import

* Fixed copy commands

* Update tokenization_roformer.py

* remove roberta fast tokenizer dependency

* fix flake8

* Update src/transformers/models/longformer/tokenization_longformer_fast.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update configuration_bart.py

* Update documentation_tests.txt

* Update documentation_tests.txt

Putting this line in a sorted order

* Adding links to pipelines parameters documentation

Adding PR based on suggestion in this issue https://github.com/huggingface/transformers/issues/19038#issuecomment-1259592359

* styling

* Updated config.yml

* Updated config.yml

* update README_es.md

* Add initial files for depth estimation pipelines

* Add test file for depth estimation pipeline

* Update model mapping names

* Add updates for depth estimation output

* Add generic test

* Hopefully fixing the tests.

* Check if test passes

* Add make fixup and make fix-copies changes after rebase with main

* Rebase with main

* Fixing up depth pipeline.

* This is not used anymore.

* Fixing the test. `Image` is a module `Image.Image` is the type.

* Update docs/source/en/main_classes/pipelines.mdx
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fixed typo for SQuAD

* Fixed the preprocess_validation_function function for the labels to reflect the remaining truncated instances

* Rolled back the trainer_seq2seq_qa.py for UnboundLocalError: local variable 'metrics' referenced before assignment
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* First draft

* Fix more things

* Improve more things

* Remove some head models

* Fix more things

* Add missing layers

* Remove tokenizer

* Fix more things

* Fix copied from statements

* Make all tests pass

* Remove print statements

* Remove files

* Fix README and docs

* Add integration test and fix organization

* Add tips

* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Make tests faster, improve docs

* Fix doc tests

* Add model to toctree

* Add docs

* Add note about creating new checkpoint

* Remove is_decoder

* Make tests smaller, add docs
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* BertConfig for doctest

* Change import order
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Change the depth to 20

* Add comment
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* Add multi-node conditions in trainer_qa.py and trainer_seq2seq.py

* Code improvement

* Generate config on the file

* Fake modif for all test launch

* Upload more artifacts

* Typo and quality

* Try converting th yml to txt

* Leave my long lines alone yaml

* Debug prints

* Debug prints v2

* Try without sorting

* Was it really working before?

* Typo

* Use a parameter

* Use a parameter?

* Typo

* Here is some JSON

* Another try

* Learning to read...

* Check default is used

* Does this work?

* With continuation

* WiP

* Use a parameter for test list

* Other fake modif

* With the comma

* Name the test step so it doesn't blow up

* Just one example modification

* Final steps

* Add nightlies

* Move config generator

* Add trigger for nightlies

* Better workflow

* Rebase on recent changes

* Fix config creation

* Fake modif in an example

* Now fake modif in one config file

* Fix install step in custom tokenizers test

* Fix generated config

* Better fix hopefully

* Finally test modif in setup

* final cleanup

* Fix doc example for OPTForQuestionAnswering
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Update PT to TF CLI model inputs

* Get padding strategy if specified

* Make False comparison explicit

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* implemented TFCvtModel and TFCvtForImageClassification and modified relevant files, added an exception in convert_tf_weight_name_to_pt_weight_name, added quick testing file to compare with pytorch model

* added docstring + testing file in transformers testing suite

* added test in testing file, modified docs to pass repo-consistency, passed formatting test

* refactoring + passing all test

* small refacto, removing unwanted comments

* improved testing config

* corrected import error

* modified acces to pretrained model archive list, to pass tf_test

* corrected import structure in init files

* modified testing for keras_fit with cpu

* correcting PR issues + Refactoring

* Refactoring : improving readability and reducing the number of permutations

* corrected momentum value + cls_token initialization

* removed from_pt as weights were added to the hub

* Update tests/models/cvt/test_modeling_tf_cvt.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Adding the README_es.md and reference to it in the others files readme

* Updating the check_copies.py

* Updating README_es.md

* Updating chec_copies

* Added tokenize keyword arguments to feature extraction pipeline

* Reverted truncation parameter

* Import numpy moved to top

* Make cpm tokenization independent of xlnet

* Make bert japanese tokenization independent of bert

🚨🚨🚨  TF: Remove `TFWrappedEmbeddings` (breaking: TF embedding initialization updated for encoder-decoder models) (#19263)

* added test

* correct embedding init

* some changes in blenderbot (incomplete)

* update blenderbot (diff to be used as reference)

* update blenderbot_small

* update LED

* update marian

* update T5 and remove TFWrappedEmbeddings

* nullcontext() -> ContextManagers()

* fix embedding init

* fix device mismatch

* make fixup

* added slow tests

- added slow tests on `bnb` models to make sure generate works correctly

* replace with `self.device`

* revert force device assign

* Update src/transformers/generation_utils.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* set the warning in `generate` instead of `sample`
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Implement multiple span support

* Address comments

* Add tests + fix bugs

* decouples xlm_prophet from prophet and adds copy patterns that pass the copy check

* adds copy patterns to copied docstrings too

* restores autodoc for XLMProphetNetModel

* removes all-casing in a bunch of places to ensure that the model is compatible with all checkpoints on the hub

* adds missing model to main init

* adds autodocs to make document checker happy

* adds missing pretrained model import

* adds missing pretrained model import to main init

* adds XLMProphetNetPreTrainedModel to the dummy pt objects

* removes examples from the source-doc file since docstrings contain them already

* adds a missing new line to make check_repo happy

* cast positions dtype in XGLMModel

* Get the correct dtype at init time

* Get the correct dtype at init time
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* remove config dependence

* remove dependencies from xlm_roberta

* Fix style

* Fix comments

* various fixes

* Fix pre-trained model name

* fixing tokenizer

* adding all missing classes

* fast tokenizer | fixing format

* revert to full class copy flag

* fixing different casing

* correct embedding init

* [Swin] Replace hard-coded batch size to enable dynamic ONNX export

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>