GitLab

* update feature extractor params

* update attention mask handling

* fix doc and pipeline test

* add warning when skipping test

* add whisper translation and transcription test

* fix build doc test

* Custom TF signature draft

* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Adding tf signature tests

* Fixing signature check and adding asserts

* fixing model load path

* Adjusting signature tests

* Formatting file
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Dimitre Oliveira <dimitreoliveira@Dimitres-MacBook-Air.local>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Fixed a non-working hyperlink in the README.md file

The hyperlink to the community notebooks was outdated.

* Fixing missing double slash in hyperlink

The momentum value for PyTorch and TensorFlow batch normalization layers is not equivalent. The TensorFlow value should be (1 - pytorch_momentum) in order to ensure the correct updates are applied to the running mean and running variance calculations. We wouldn't observe a difference loading a pretrained model and performing inference, but evaluation outputs would change after some training steps.

* fix conflicts

* start translating

* proof check

* add toc

* fix errors and typos

* simplify loop

* add featur extractor

* add model

* start conversion

* add dropout

* initial commit of test files

* copnversion for all models

* update processor for correct padding

* update feature extraction

* update integration test logits match

* fmnt: off for the logits

* on the fly mel bank

* small nit

* update test

* update tokenizer

* nit feature extraction

* update

* update tokenizer test

* adds logit processor and update tokenizer to get supress tokens

* style

* clean convert

* revert to original modeling tf utils

* Update

* update

* nit

* clean convert file

* update tests and nits

* quality

* slow generation test

* ffn_dim to allow customization

* update readme

* add to toctreee

* start fixing integration tests

* update tests and code

* fix feature extractor

* fix config tests common

* update code to fix tests

* fix feature exctractor

* nit feature extractio...

* Add `OPTForQuestionAnswering`

- added `OPTForQuestionAnswering` class based on `BloomForQuestionAnswering`
- added `OPTForQuestionAnswering` in common tests
- all common tests pass
- make fixup done

* added docstrings for OPTForQuestionAnswering

* Fix docstrings for OPTForQuestionAnswering

The sequence_masked variable is actually the part of the sequence that is kept unmasked for the encoder. This commit renames the variable.

* Remove dependency of Roberta in Blenderbot

* Move Copied from statements to each method of the Roberta classes

* Remove copied from line for mask_token.setter

* update output from example in docs

* Add onnx support for VisionEncoderDecoder

* Add onnx support for VisionEncoderDecoder

* Removed unused import

* Rename encoder hidden state
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* Update docstrings and removed redundant code

* Added test function for enc-dec models

* Update doc string text
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* fixed code style
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* Leverage hfh for move cache

* Style

Fix link typo of the following content.
PyTorch version, Trainer
PyTorch version, no Trainer

* fix marianMT convertion to onnx

* Update src/transformers/onnx/convert.py
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* Update src/transformers/onnx/convert.py
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* Fixed duplicated line (paragraph #83) @omarespejel @sgugger

* Datasets map denomination fixed (paragraph 42)

* remove RobertaConfig inheritance from MarkupLMConfig

* Update src/transformers/models/markuplm/configuration_markuplm.py

fixed typo in docstring
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Rework pipeline tests

* Try to fix Flax tests

* Try to put it before

* Use a new decorator instead

* Remove ignore marker since it doesn't work

* Filter pipeline tests

* Woopsie

* Use the fitlered list

* Clean up and fake modif

* Remove init

* Revert fake modif

- Fixes the image segmentation pipeline test failures caused by changes to the postprocessing methods of supported models
- Updates the ImageSegmentationPipeline tests
- Improves docs, adds 'task' argument to optionally perform semantic, instance or panoptic segmentation

* Copied all the code required from transformers.models.bert.modeling_bert to here

* Fixed styling issues

* Reformatted copied names with Model specific name.

* Reverted BertEncoder part as there is already a class called BertGenerationEncoder

* Added prefixes in missing places.
Co-authored-by: vishwaspai <vishwas.pai@emplay.net>

* camembert tf version independent

* fixup

* fixup, all working

* remove comments

* Adding copied from roberta
Co-authored-by: Mustapha AJEGHRIR <mustapha.ajeghrir@kleegroup.com>

* Copied from BertTokenizer() in tokenization_bert

* Added BasicTokenizer and WordPieceTokenizer Class

* Update src/transformers/models/electra/tokenization_electra.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Added copied from comments for basicTokenizer and WordPieceTokenizer

* Updated the comments for the tokenizerClasses

* Update src/transformers/models/electra/tokenization_electra.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/electra/tokenization_electra.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Formatted tokenization_electra with `make style`

* Fix repo inconsistencies

* Update src/transformers/models/electra/tokenization_electra.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Set the logger
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* removed dependency from bart(slow)

* removed dependency from bart(slow)

* adding copying comments (copied from bart to led)

* updated led docstring

* updated led docstring

* removed dependency from Bart (fast)

* replaced bart with LED in docstrings

* complying flake8

* added more copy comments

* fixing copying comments

* added comments back

* fix copy comments

* fixing copied from comments

* fixing copied from comments

* add first generation tutorial

* uP

* [Clip] Add text model to device map

Co-authored-by: harry7337 <hari.8jan@gmail.com>

* Remove dependency of Bert from Squeezebert tokenizer

* run style corrections

* update copies from BertTokenizers

* Update changes and style to Squeezebert files

* update copies for bert-fast