Commits · dad5ca83b2f7303ffd3e39c33dbe05f79ac56571 · chenpangpang / transformers

"docs/source/ms/index.md" did not exist on "900677487d8953fa1fabdc3e298abe63ff09ece8"

04 Apr, 2022 8 commits
- TF: Finalize `unpack_inputs`-related changes (#16499) · dad5ca83
  Joao Gante authored Apr 04, 2022
```
* Add unpack_inputs to remaining models

* removed kwargs to `call()` in TF models

* fix TF T5 tests
```
  dad5ca83
- add a test checking the format of `convert_tokens_to_string`'s output (#16540) · be9474bd
  SaulLu authored Apr 04, 2022
```
* add new tests

* add comment to overridden tests
```
  be9474bd
- Add use_auth to load_datasets for private datasets to PT and TF examples (#16521) · 24a85cca
  Karim Foda authored Apr 04, 2022
```
* fix formatting and remove use_auth

* Add use_auth_token to Flax examples
```
  24a85cca
- Enable doc in Spanish (#16518) · b9a768b3
  Sylvain Gugger authored Apr 04, 2022
```
* Reorganize doc for multilingual support

* Fix style

* Style

* Toc trees

* Adapt templates
```
  b9a768b3
- Add utility to find model labels (#16526) · 3951b9f3
  Sylvain Gugger authored Apr 04, 2022
```
* Add utility to find model labels

* Use it in the Trainer

* Update src/transformers/utils/generic.py
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* Quality
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
```
  3951b9f3
- Fix flax import in __init__.py: modeling_xglm -> modeling_flax_xglm (#16556) · ec4da72f
  Daniel Stancl authored Apr 04, 2022
  
  ec4da72f
- Making the impossible to connect error actually report the right URL. (#16446) · 013a7dbe
  Nicolas Patry authored Apr 04, 2022
  
  013a7dbe
- [FlaxSpeechEncoderDecoder] Fix dtype bug (#16581) · ad0cba08
  Patrick von Platen authored Apr 04, 2022
```
* [FlaxSpeechEncoderDecoder] Fix dtype bug

* more fixes
```
  ad0cba08
01 Apr, 2022 12 commits
- Add code samples for TF speech models (#16494) · 60d27b1f
  Yih-Dar authored Apr 01, 2022
```
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
  60d27b1f
- Pin tokenizers version <0.13 (#16539) · 53a4d6b1
  Lysandre Debut authored Apr 01, 2022
```
* Pin tokenizers version <0.13

* Style
```
  53a4d6b1
- Improve code example (#16450) · 61ee26a8
  NielsRogge authored Apr 01, 2022
```
Co-authored-by: Niels Rogge <nielsrogge@nielss-mbp.home>
```
  61ee26a8
- Use random_attention_mask for TF tests (#16517) · 2199382d
  Yih-Dar authored Apr 01, 2022
```
* use random_attention_mask for TF tests

* Fix for TFCLIP test (for now).
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
  2199382d
- Remove MBart subclass of XLMRoberta in tokenzier docs (#16546) · 823dbf8a
  Gunjan Chhablani authored Apr 01, 2022
```
* Remove MBart subclass of XLMRoberta in tokenzier

* Fix style

* Copy docs from MBart50 tokenizer
```
  823dbf8a
- Adding missing type hints for mBART model (PyTorch) (#16429) · 5fe06b9b
  Rishav Chandra Varma authored Apr 01, 2022
```
* added type hints for mbart tensorflow tf implementation

* Adding missing type hints for mBART model 

Tensorflow Implementation model added with missing type hints

* Missing Type hints - correction

For TF model

* Code fixup using make quality tests

* Hint types - typo error

* make fix-copies and make fixup

* type hints

* updated files

* type hints update

* making dependent modesls coherent
Co-authored-by: matt <rocketknight1@gmail.com>
```
  5fe06b9b
- Add VisualBert type hints (#16544) · 9947dd07
  Gunjan Chhablani authored Apr 01, 2022
  
  9947dd07
- Fix Bart type hints (#16297) · 59a9c83e
  Gunjan Chhablani authored Apr 01, 2022
```
* Add type hints to PLBart PyTorch

* Remove pending merge conflicts

* Fix PLBart Type Hints

* Add changes from review
```
  59a9c83e
- Type hints added (#16529) · afc5a1ea
  Dahlbomii authored Apr 01, 2022
  
  afc5a1ea
- call on_train_end when trial is pruned (#16536) · 483a9450
  Ferdinand Schlatt authored Apr 01, 2022
  
  483a9450
- Add ONNX export for BeiT (#16498) · 9de70f21
  Jim Rohrer authored Apr 01, 2022
```
* Add beit onnx conversion support

* Updated docs

* Added cross reference to ViT ONNX config
```
  9de70f21
- Fixed a typo in legacy seq2seq_trainer.py (#16531) · bfeff6cc
  Cathy authored Apr 01, 2022
  
  bfeff6cc
31 Mar, 2022 9 commits

[research] link to the XTREME-S paper (#16519) · 5807054b

Anton Lozhkov authored Mar 31, 2022



* [research] link to the XTREME-S paper

* Update examples/research_projects/xtreme-s/README.md
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

5807054b

Fix syntax error in generate docstrings (#16516) · e4b23483
Sylvain Gugger authored Mar 31, 2022

e4b23483

added type hints to xglm pytorch (#16500) · b808d8a5

Mowaninuola Osifeso authored Mar 31, 2022



* added type hints to xglm pytorch

* Update src/transformers/models/xglm/modeling_xglm.py

* Update src/transformers/models/xglm/modeling_xglm.py
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

b808d8a5

fixed a typo (#16508) · 05b4c329
Bhadresh Savani authored Mar 31, 2022

05b4c329

Translate accelerate.mdx from english to spanish (#16176) · 6a4dbba1

Santiago Gómez authored Mar 31, 2022



* Translate accelerate.mdx from english to spanish

* Update docs/source_es/accelerate.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Apply suggestions from code review
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Apply suggestions from code review
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Fix nits and finish translation
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

6a4dbba1

Translate installation.mdx to Spanish (#16229) · c551adde

Liliana Badillo authored Mar 31, 2022



* Translate installation.mdx to Spanish

* Update docs/source_es/installation.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/installation.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/installation.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/installation.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/installation.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/installation.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/installation.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/installation.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/installation.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/installation.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Fix nits and finish translation
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

c551adde

Spanish translation of the file multilingual.mdx (#16329) · 98939e6a

Juanjo do Olmo authored Mar 31, 2022



* Duplication of the source eng file

* Spanish translation of the file multilingual.mdx

* Update docs/source_es/multilingual.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/multilingual.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/multilingual.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/multilingual.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/multilingual.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/multilingual.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Update docs/source_es/multilingual.mdx
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

* Fix nits and finish translation
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

98939e6a

make tuple annotation more specific to avoid failures during symbolic_trace (#16490) · 99a01423

chenbohua3 authored Mar 31, 2022

* make tuple annotation more specific to avoid failures during symbolic_trace

* make tuple annotation more specific to avoid failures during symbolic_trace

99a01423

Refactor Modeling Outputs (#16341) · a8b6443e

Francesco Saverio Zuppichini authored Mar 31, 2022



* first proposal

* replace model outputs in various models

* conflicts

* docstring

* update poolformer

* minor change in docstring

* CI

* removed poolformer specific outputs from doc

* removed convnext specific outputs from doc

* CI

* weird char in segformer

* conversations

* reverted docstring for BaseModelOutputWithPooling

* update outputs

* changed docstring in BaseModelOutput

* updated docstring in modeling outputs

* typos :)

* fixed typo after copy & paste it all around

* CI

* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* segformer
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

a8b6443e

30 Mar, 2022 9 commits

Support reduce_bucket_size=auto for deepspeed stages <3 (#16496) · 857eb87c
Manuel R. Ciosici authored Mar 30, 2022

857eb87c

update smddp api to v1.4.0 (#16371) · 81ac45f8

Lai Wei authored Mar 30, 2022



* update smddp api to v1.4.0

* Update src/transformers/trainer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/trainer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* address comments

* fix style

* remove unused import

* fix indent

* disable style check for import

* fix space
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

81ac45f8

[examples] max samples can't be bigger than the len of dataset (#16501) · a73281e3
Stas Bekman authored Mar 30, 2022
```
* [examples] max samples can't be bigger than then len of dataset

* do tf and flax
```
a73281e3

Feature Extractor accepts `segmentation_maps` (#15964) · c4deb7b3

Francesco Saverio Zuppichini authored Mar 30, 2022



* feature extractor accepts

* resolved conversations

* added examples in test for ADE20K

* num_classes -> num_labels

* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* resolving conversations

* resolving conversations

* removed ADE

* CI

* minor changes in conversion script

* reduce_labels in feature extractor

* minor changes

* correct preprocess for instace segmentation maps

* minor changes

* minor changes

* CI

* debugging

* better padding

* going to update labels inside the model

* going to update labels inside the model

* minor changes

* tests

* removed changes in feature_extractor_utils

* conversation

* conversation

* example in feature extractor

* more docstring in modeling

* test

* make style

* doc
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

c4deb7b3

TF: unpack inputs on Convbert, GPTJ, LED, and templates (#16491) · c2f8eaf6

Joao Gante authored Mar 30, 2022

* Add unpack_inputs to remaining models

* remove stray use of inputs in the templates; fix tf.debugging of attn masks

c2f8eaf6

Add support for exporting GPT-J to ONNX-TRT (#16492) · ae189ef9
tomerip authored Mar 30, 2022
```
Add support for exporting GPT-J to ONNX-TRT
Co-authored-by: Tomer Stav <stavt@amazon.com>
```
ae189ef9
Add length to PreTrainedTokenizer train_new_from_iterator (#16493) · d04adc35
dctelus authored Mar 30, 2022

d04adc35
Nit: MCSCOCO -> MS COCO (#16481) · 147c8166
Aditya Kane authored Mar 30, 2022

147c8166

TF GPT-J Type hints and TF decorator (#16488) · ffd19ee1

Dahlbomii authored Mar 30, 2022



* Type hints and TF decorator added

* Type hints and TF decorator added

* make style
Co-authored-by: matt <rocketknight1@gmail.com>

ffd19ee1

29 Mar, 2022 2 commits
- Do not initialize `torch.distributed` process group if one is already initailized (#16487) · 277d49a5
  Antoni Baum authored Mar 29, 2022
```
* Do not initialize torch process group twice

* Apply suggestions from code review
```
  277d49a5
- Raise diff tolerance value for TFViTMAEModelTest (#16483) · 2b483230
  Yih-Dar authored Mar 29, 2022
```
* Raise diff tolerance value
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
```
  2b483230