GitLab

* add return_tensors parameter for feature_extraction  w/ test

add return_tensor parameter for feature extraction

Revert "Merge branch 'feature-extraction-return-tensor' of https://github.com/ajsanjoaquin/transformers

 into feature-extraction-return-tensor"

This reverts commit d559da743b87914e111a84a98ba6dbb70d08ad88, reversing
changes made to bbef89278650c04c090beb65637a8e9572dba222.

* call parameter directly
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>

* Fixup.

* Update src/transformers/pipelines/feature_extraction.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

install->installation

* fix warnings in deberta

* fix copies

* Revert "fix copies"

This reverts commit 324cb3fed11e04190ba7b4662644baa8143b60be.

* fix copies

* fix copies again

* revert changes to whitespace that make style did since it results in an infinite chain of fix-copies

* argh
Co-authored-by: Sander Land <sander@chatdesk.com>

* Removed Bert interdependency from Funnel transformer

* passed consistency check

* Revert "passed consistency check"

This reverts commit ba55a0813549938fc54626794e666ee13a85c2d8.

* Fixed docstrings
Co-authored-by: mukesh663 <mukesh13034@gmail.com>

* Fixes

* update expected values

* style

* fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* added type hints for Yolos Pytorch model

* make fixup

* Update src/transformers/models/yolos/convert_yolos_to_pytorch.py

* Update src/transformers/models/yolos/convert_yolos_to_pytorch.py

* Update src/transformers/models/yolos/convert_yolos_to_pytorch.py
Co-authored-by: Matt <rocketknight1@gmail.com>
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* Partial TF port for ESM model

* Add ESM-TF tests

* Add the various imports for TF-ESM

* TF weight conversion almost ready

* Stop ignoring the decoder weights in PT

* Add tests and lots of fixes

* fix-copies

* Fix imports, add model docs

* Add get_vocab() to tokenizer

* Fix vocab links for pretrained files

* Allow multiple inputs with a sep

* Use EOS as SEP token because ESM vocab lacks SEP

* Correctly return special tokens mask from ESM tokenizer

* make fixup

* Stop testing unsupported embedding resizing

* Handle TF bias correctly

* Skip all models with slow tokenizers in the token classification test

* Fixing the batch/unbatcher of pipelines to accomodate the `None` being

passed around.

* Fixing pipeline bug caused by slow tokenizer  being different.

* Update src/transformers/models/esm/modeling_tf_esm.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Update src/transformers/models/esm/modeling_tf_esm.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Update src/transformers/models/esm/modeling_tf_esm.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Update set_input_embeddings and the copyright notices
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* add type hints to mctct

* run auto style corrections

* change torch.bool to bool#

* Update src/transformers/models/mctct/modeling_mctct.py
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* Remove optional tags for attention_mask and head_mask'

* fix optional tags'

* Update src/transformers/models/mctct/modeling_mctct.py
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* Remove key word argument X from pipeline predict and transform methods

As __call__ of pipeline clasees require one positional argument, passing
the input as a keyword argument inside predict, transform methods, causing
__call__ to fail. Hence in this commit the keyword argument is modified
into positional argument.

* Implement basic tests for scikitcompat pipeline interface

* Seperate tests instead of running with parameterized based on framework as both frameworks will not be active at the same time

* trocr Config for doctest

* ran make style

* Data2Vec Text Config for doctest

* typo fix

* made suggested changes

* initial changes

* update the suggested order of import

* adds vision_encoder_decoder to Doc tests

* keep the initial order

* initial commit

* few suggested changes

* simplify loop

* fix layer map split

* update

* update for special variables

* add rag test

* fixup

* revert change : for next PR

* update feature extractor params

* update attention mask handling

* fix doc and pipeline test

* add warning when skipping test

* add whisper translation and transcription test

* fix build doc test

* Correct whisper processor

* make fix copies

* remove sample docstring as it does not fit whisper model

* Update src/transformers/models/whisper/modeling_whisper.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix, doctests are passing

* Nit

* last nit
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* ResNet Config for doctest

* added empty lines as suggested

* ran make style

* [Whisper] Fix gradient checkpointing (again!)

* [Whisper] Fix checkpointing (again!)

* Improve error messaging for ASR pipeline.

- Raise error early (in `_sanitize`) so users don't waste time trying to
  run queries with invalid params.

- Fix the error was after using `config.inputs_to_logits_ratio` so our
  check was masked by the failing property does not exist.

- Added some manual check on s2t for the error message.
  No non ctc model seems to be used by the default runner (they are all
  skipped).

* Removing pdb.

* Stop the early error it doesn't really work :(.

* fix: small error

* fix: another typo error

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* GPTTOkenizer dependency removed from deberta class

Fixup

made the Deberta Tokenizer fast independent of GPT-2 tokenizer

Copied annotation added

Done the dependency removal

* Added some missing copied statement

* Added some copied statements

* fix flaubert tokenizer

* update

* update

* Final cleanup
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>