GitLab

* Fix tokenizer load from one file

* Add a test

* Style
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>

* added type hints pytorch unispeech

* added type hints pytorch  MPNet

* added type hints nystromformer

* resolved copy inconsistencies

* make fix-copies
Co-authored-by: matt <rocketknight1@gmail.com>

* resized models that we can actually load

* separate embeddings check

* add test for embeddings out of bounds

* add fake slows

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Add CLIP to zero-shot-image-classification

* Make mapping private as it's not used for AutoClassing

* Tests conditional run

* Syntax

* Deps

* Try early exit

* Another way

* Test with no tests to run

* Test all

* Typo

* Try this way

* With tests to run

* Mostly finished

* Typo

* With a modification in one file only

* No change, no tests

* Final cleanup

* Address review comments

* Adding cast to fp32 in convnext layernorm to prevent rounding errors in the case of fp16 input

* Trigger CI

* Support for ConvNext

* Support for Wav2Vec2

* Support for Resnet

* Fix small issue in test_modeling_convnext

- set `use_cache` to `True` for consistency with other `transformers` models

* Adds package and requirement spec output to version check exception

It's difficult to understand what package is affected when `got_ver`
here comes back None, so output the requirement and the package. The
requirement probably contains the package but let's output both for good
measure.

Non-exhaustive references for this problem aside from my own encounter:

* https://stackoverflow.com/questions/70151167/valueerror-got-ver-is-none-when-importing-tensorflow
* https://discuss.huggingface.co/t/valueerror-got-ver-is-none/17465
* https://github.com/UKPLab/sentence-transformers/issues/1186
* https://github.com/huggingface/transformers/issues/13356

I speculate that the root of the error comes from a conflict of
conda-managed and pip-managed Python packages but I've not yet proven
this.

* Combines version presence check and streamlines exception message

See also: https://github.com/huggingface/transformers/pull/18702#discussion_r953223275

Co-authored-by: Stas Bekman <stas@stason.org>

* Enable torchdynamo tests

* make style
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Fix CI for custom tokenizers

* Add nightly tests

* Run CI, run!

* Fix paths

* Typos

* Fix test

* Fixing OPT fast tokenizer option.

* Remove dependency on `pt`.

* Move it to GPT2 tokenization tests.

* Added a few tests.

* init PR

* optimize top p and add edge case

* styling

* style

* revert tf and flax test

* add edge case test for FLAX and TF

* update doc with smallest set sampling for top p

* make style

* Override save() to use the serving signature as the default

* Replace int32 with int64 in all our serving signatures

* Remember one very important line so as not to break every test at once

* Dtype fix for TFLED

* dtype fix for shift_tokens_right in general

* Dtype fixes in mBART and RAG

* Fix dtypes for test_unpack_inputs

* More dtype fixes

* Yet more mBART + RAG dtype fixes

* Yet more mBART + RAG dtype fixes

* Add a check that the model actually has a serving method

* Fix test_save_load for TFViTMAEModelTest
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Updated test values

The image segmentation pipeline tests - tests/pipelines/test_pipelines_image_segmentation.py - were failing after the merging of #1849  (49e44b21). This was due to the difference in rescaling. Previously the images were rescaled by `image = image / 255`. In the new commit, a `rescale` method was added, and images rescaled using `image = image * scale`. This was known to cause small differences in the processed images (see
[PR comment](https://github.com/huggingface/transformers/pull/18499#discussion_r940347575)).

Testing locally, changing the `rescale` method to divide by a scale factor (255) resulted in the tests passing. It was therefore decided the test values could be updated, as there was no logic difference between the commits.

* Use double quotes, like previous example

* Fix up

* Small replacement

- replace `modules_to_not_convert` by `module_to_not_convert`

* refactor a bit

- changed variables name
- now output a list
- change error message

* make style

* add list

* make style

* change args name
Co-authored-by: stas00 <stas00@users.noreply.github.com>

* fix comment

* fix typo
Co-authored-by: stas00 <stas00@users.noreply.github.com>

* Update src/transformers/modeling_utils.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: stas00 <stas00@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

correct the import statement

* Fix a broken link for deepspeed ZeRO inference

* fix link
Co-authored-by: Stas Bekman <stas@stason.org>

* Automate check for new pipelines and metadata update

* Add Datasets to quality extra

* add gpt-neox-japanese model and tokenizer as new model

* Correction to PR's comment for GPT NeoX Japanese
- Fix to be able to use gpu
- Add comment # Copied... at the top of RotaryEmbedding
- Implement nn.Linear instead of original linear class
- Add generation test under @slow

* fix bias treatment for gpt-neox-japanese

* Modidy gpt-neox-japanese following PR
- add doc for bias_dropout_add
- style change following a PR comment

* add document for gpt-neox-japanese

* remove unused import from gpt-neox-japanese

* fix README for gpt-neox-japanese

* Fix DocumentQuestionAnsweringPipelineTests
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Skip test_torchscript_output_attentions for PegasusXModelTest

* fix test_inference_no_head

* fix test_inference_head

* fix test_seq_to_seq_generation
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Clarify cookiecutter questions

* Update first question
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>