GitLab

with pytorch cpu only version. without --no_cuda, using --bf16 will trigger error like "Your setup doesn't support bf16/gpu. You need torch>=1.10, using Ampere GPU with cuda>=11.0" (#20445)

make tensors in function build_relative_position created on proper device instead of always on cpu (#20434)
Co-authored-by: wenhanli <wenhanli@tencent.com>

* Add a test to ensure int dummy inputs are int64

* Move the test into the existing int64 test and update a lot of existing dummies

* Fix remaining dummies

* Fix remaining dummies

* Test for int64 serving sigs as well

* Update core tests to use tf.int64

* Add better messages to the assertions

* Update all serving sigs to int64

* More sneaky hiding tf.int32s

* Add an optional int32 signature in save_pretrained

* make fixup

* Add Amy's suggestions

* Switch all serving sigs back to tf.int32

* Switch all dummies to tf.int32

* Adjust tests to check for tf.int32 instead of tf.int64

* Fix base dummy_inputs dtype

* Start casting to tf.int32 in input_processing

* Change dtype for unpack_inputs test

* Add proper tf.int32 test

* Make the alternate serving signature int64

* Re-enable TF ESM tests, make sure we use facebook checkpoints

* make fixup

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Fix typo

* Update tokenization_fsmt.py

* fix docstring

* fix 2

* add details

* `accelerate` support for `OwlViT`

- added `accelerate` support
- added slow `fp16` tests

* apply suggestions

* Update _toctree and clone original doc

* Forgot to translate (lol)

* Translate documentation and update toctree

* Add suggested changes from review

* Use updated values
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* `bnb` ValueError when tries to cast or assign

* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* remove docstrings

* change error log
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* change the way sentinel tokens can retrived

* Fix line length for doc string

* Fix line length for doc string

* Add more stronger test for t5 tokenization

* Format file changes

* Make a stronger test for filtering sentinel tokens

* fix file format issues

* Correct type checking

* up

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* [bloom] convert script tweaks

* Update src/transformers/models/bloom/convert_bloom_original_checkpoint_to_pytorch.py
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* improve the 2nd assert

* add conversion readme
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* fix `accelerate` tests for esmfold

* cleaner solution

* Use tiny ONNX models

* Fix broken tests

* Add tiny perceiver

* Add tiny convbert

* fix controlflow for onnx export

* fix warning

* fix the case padding_len = 0, explicit the recorded control flows

* style

* style

* fix bug

* fix copy

* nits

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

* Optimizes DonutProcessor token2json method for speed

* Applies black formatting

* Updates Donut pretrained model name in test file

* remaining pytorch type hints (#20217)

* Update modeling_flava.py

* Update modeling_markuplm.py

* Update modeling_glpn.py

* Update modeling_roc_bert.py

* Update modeling_segformer.py

* Update modeling_tapas.py

* Update modeling_tapas.py

* Update modeling_tapas.py

* Update modeling_tapas.py

* Update modeling_trocr.py

* Update modeling_videomae.py

* Update modeling_videomae.py

* Update modeling_videomae.py

* Update modeling_yolos.py

* Update modeling_wav2vec2.py

* Update modeling_jukebox.py

* Update modeling_jukebox.py

* Update modeling_jukebox.py

* Update modeling_jukebox.py

* Data collator for token classification pads labels column when receives pytorch tensors (#20244)

* token cls data_collator pads labels column

* remove walrus operator for code quality

* remove redundat space

* remove comment that was fixed

* PR comments fix
Co-authored-by: Alexander Markov <amarkov.me@gmail.com>

* [Doctest] Add configuration_deformable_detr.py (#20273)

* Update configuration_deformable_detr.py comment

* Add DeformableDetrConfig to documentation_tests.txt

* Fix summarization script (#20286)

* [DOCTEST] Fix the documentation of RoCBert (#20142)

* update part of the doc

* add temp values, fix part of the doc

* add template outputs

* add correct models and outputss

* style

* fixup

* [bnb] Let's warn users when saving 8-bit models (#20282)

* add warning on 8-bit models

- added tests
- added wrapper

* move to a private attribute

- remove wrapper
- changed `save_pretrained` method

* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix suggestions
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Adding `zero-shot-object-detection` pipeline doctest. (#20274)

* Adding `zero-shot-object-detection` pipeline doctest.

* Remove nested_simplify.

* Adding doctest for `object-detection` pipeline. (#20258)

* Adding doctest for `object-detection` pipeline.

* Removed nested_simplify.

* Image transforms functionality used instead (#20278)

* Image transforms functionality used instead

* Import torch

* Import rather than copy

* Update src/transformers/models/conditional_detr/feature_extraction_conditional_detr.py

* TF: add test for `PushToHubCallback` (#20231)

* test hub tf callback

* create repo before cloning it

* Generate: general TF XLA constrastive search are now slow tests (#20277)

* move contrastive search test to slow

* Fixing the doctests failures. (#20294)

* Fixing the doctests failures.

* Fixup.

* set the default cache_enable to True, aligned with the default value in pytorch cpu/cuda amp autocast (#20289)
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* Add docstrings for canine model (#19457)

* Add docstrings for canine model

* Update CanineForTokenClassification
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Add AutoBackbone + ResNetBackbone (#20229)

* Add ResNetBackbone

* Define channels and strides as property

* Remove file

* Add test for backbone

* Update BackboneOutput class

* Remove strides property

* Fix docstring

* Add backbones to SHOULD_HAVE_THEIR_OWN_PAGE

* Fix auto mapping name

* Add sanity check for out_features

* Set stage names based on depths

* Update to tuple
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

* Add missing report button for Example test (#20293)
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* refactor test (#20300)

- simplifies the devce checking test

* [Tiny model creation] deal with `ImageProcessor` (#20298)
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Fix blender bot missleading doc (#20301)

* fix the doc to specify that add_prefix_space = False

* add correct expected output

* remove two tokens that should not be suppressed (#20302)

* [ASR Examples] Update README for Whisper (#20230)

* [ASR Examples] Update README for seq2seq

* add language info

* add training results

* re-word

* Add padding image transformation (#19838)

* Add padding transformation

* Add in upstream changes

* Update tests & docs

* Code formatting tuples in docstring

* Pin TensorFlow (#20313)

* Pin to the right version...

* Also pin TensorFlow CPU

* Add AnyPrecisionAdamW optimizer (#18961)

* Add AnyPrecisionAdamW optimizer

* Add optim_args argument to TrainingArgs

* Add tests for AnyPrecisionOptimizer

* Change AnyPrecisionAdam default params to float32

* Move default_anyprecision_kwargs in trainer test

* Rename AnyPrecisionAdamW

* [Proposal] Breaking change `zero-shot-object-detection` for improved     consistency. (#20280)

* [Proposal] Breaking change `zero-shot-object-detection` for improved
consistency.

This is a proposal to modify the output of `zero-shot-object-detection`
to provide better alignment with other pipelines.

The output is now strictly the same as `object-detection` whereas before
it would output lists of lists.

The name `candidate_labels` is used throughout for consistency with
other `zero-shot` pipelines.

The pipeline is changed to `ChunkPipeline` to support batching cleanly.

This removes all the lists and list of lists shenanigans, it's now a
matter of the base pipeline handling all this not this specific one.

**Breaking change**: It did remove complex calls potentials `pipe(images = [image1, image2],
text_queries=[candidates1, candidates2])` to support only
`pipe([{"image": image1, "candidate_labels": candidates1}, {"image": image2, "candidate_labels": candidates2}])`
when dealing with lists and/or datasets.
We could keep them, but it will add a lot of complexity to the code
base, since the pipeline is rather young, I'd rather break to keep the
code simpler, but we can revert this.

**Breaking change**: The name of the argument is now `image` instead of
`images` since it expects by default only 1 image. This is revertable
like the previous one.

**Breaking change**: The types is now simplified and flattened:

`pipe(inputs) == [{**object1}, {**object2}]`
instead of the previous
`pipe(inputs) == [[{**object1}, {**object1}], [{**object2}]]`
Where the different instances would be grouped by candidate labels
within lists.
IMHO this is not really desirable, since it would output empty lists and
is only adding superflous indirection compared to
`zero-shot-object-detection`.

It is relatively change free in terms of how the results, it does change
computation however since now the batching is handled by the pipeline
itself. It **did** change the results for the small models so there
seems to be a real difference in how the models handle this.

* Fixing the doctests.

* Behind is_torch_available.

* Fix flakey test with seed (#20318)

* Pin TF 2.10.1 for Push CI (#20319)
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Remove double brackets (#20307)

* remove double brackets

* oops get other bracket

* TF: future proof our keras imports (#20317)

* future proof our tf code

* parse tf versions

* Add Neighborhood Attention Transformer (NAT) and Dilated NAT (DiNAT) models (#20219)

* Add DiNAT

* Adds DiNAT + tests

* Minor fixes

* Added HF model

* Add natten to dependencies.

* Cleanup

* Minor fixup

* Reformat

* Optional NATTEN import.

* Reformat & add doc to _toctree

* Reformat (finally)

* Dummy objects for DiNAT

* Add NAT + minor changes

Adds NAT as its own independent model + docs, tests
Adds NATTEN to ext deps to ensure ci picks it up.

* Remove natten from `all` and `dev-torch` deps, add manual pip install to ci tests

* Minor fixes.

* Fix READMEs.

* Requested changes to docs + minor fixes.

* Requested changes.

* Add NAT/DiNAT tests to layoutlm_job

* Correction to Dinat doc.

* Requested changes.

* organize pipelines by modality (#20306)

* Fix torch device issues (#20304)

* fix device issue
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Generate: add generation config class (#20218)
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* translate zh quicktour(#20095) (#20181)

* zh quicktour(#20095)

* add zh to doc workflow

* remove untranslation from toctree
Co-authored-by: BeifangSusu <BeifangSusu@bfss.com>

* Add Spanish translation of serialization.mdx (#20245)

* Update _toctree and clone original content

* Translate first three sections

* Add more translated chapters. Only 3 more left.

* Finish translation

* Run style from doc-builder

* Address recommended changes from reviewer

* Add LayerScale to NAT/DiNAT (#20325)

* Add LayerScale to NAT/DiNAT.

Completely dropped the ball on LayerScale in the original PR (#20219).
This is just an optional argument in both models, and is only activated for larger variants in order to provide training stability.

* Add LayerScale to NAT/DiNAT.

Minor error fixed.
Co-authored-by: Ali Hassani <ahassanijr@gmail.com>

* [Switch Transformers] Fix failing slow test (#20346)

* run slow test on GPU

* remove unnecessary device assignment

* use `torch_device` instead

* fix: "BigSicence" typo in docs (#20331)

* add MobileNetV1 model (#17799)

* add model files etc for MobileNetV2

rename files for MobileNetV1

initial implementation of MobileNetV1

fix conversion script

cleanup

write docs

tweaks

fix conversion script

extract hidden states

fix test cases

make fixup

fixup it all

remove main from doc link

fixes

fix tests

fix up

use google org

fix weird assert

* fixup

* use google organization for checkpoints

* Generate: `model_kwargs` can also be an input to `prepare_inputs_for_generation` (#20353)

* Update Special Language Tokens for PLBART (#19980)

* Update Special Language Tokens for PLBART

* fix format

* making mapping for language codes and updating tests:

* fix format

* fix consistency

* add assert to both tokenizer tests.

* fix format

* Update src/transformers/models/plbart/tokenization_plbart.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* improvin readability, setting self.tgt_lang

* fixing

* readability
Co-authored-by: jordiclive <jordiclive19@imperial.ac.uk>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Add resources (#20296)
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

* Enhance HfArgumentParser functionality and ease of use (#20323)

* Enhance HfArgumentParser

* Fix type hints for older python versions

* Fix and add tests (+formatting)

* Add changes

* doc-builder formatting

* Remove unused import "Call"

* Add Audio Spectogram Transformer (#19981)

* First draft

* Make conversion script work

* Add id2label mapping, run code quality

* Fix copies

* Add first draft of feature extractor

* Update conversion script to use feature extractor

* Make more tests pass

* Add docs

* update input_features to input_values + pad by default to max length

* Fix doc tests

* Add feature extractor tests

* Add proper padding/truncation to feature extractor

* Add support for conversion of all audioset checkpoints

* Improve docs and extend conversion script

* Fix README

* Rename spectogram to spectrogram

* Fix copies

* Add integration test

* Remove dummy conv

* Update to ast

* Update organization

* Fix init

* Rename model to AST

* Add require_torchaudio annotator

* Move import of ASTFeatureExtractor under a is_speech_available

* Fix rebase

* Add pipeline config

* Update name of classifier head

* Rename time_dimension and frequency_dimension for clarity

* Remove print statement

* Fix pipeline test

* Fix pipeline test

* Fix index table

* Fix init

* Fix conversion script

* Rename to ForAudioClassification

* Fix index table
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

* Add inference section to task guides (#18781)

* 📝 start adding inference section to task guides

* ✨ make style

* 📝

 add multiple choice

* add rest of inference sections

* make style

* add compute_metric, push_to_hub, pipeline

* make style

* add updated sequence and token classification

* make style

* make edits in token classification

* add audio classification

* make style

* add asr

* make style

* add image classification

* make style

* add summarization

* make style

* add translation

* make style

* add multiple choice

* add language modeling

* add qa

* make style

* review and edits

* apply reviews

* make style

* fix call to processor

* apply audio reviews

* update to better asr model

* make style

* Fix toctree for Section 3 in Spanish Documentation (#20360)

* Order and group topics in the right section

* Translate "Computer Vision"
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Co-authored-by: IMvision12 <88665786+IMvision12@users.noreply.github.com>
Co-authored-by: Alexander Markov <almarkv@yandex.ru>
Co-authored-by: Alexander Markov <amarkov.me@gmail.com>
Co-authored-by: Saad Mahmud <shuvro.mahmud79@gmail.com>
Co-authored-by: Zachary Mueller <muellerzr@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Wang, Yi <yi.a.wang@intel.com>
Co-authored-by: raghavanone <115454562+raghavanone@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <Sylvain.gugger@gmail.com>
Co-authored-by: atturaioe <76523524+atturaioe@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Ali Hassani <68103095+alihassanijr@users.noreply.github.com>
Co-authored-by: BFSS <31245245+bfss@users.noreply.github.com>
Co-authored-by: BeifangSusu <BeifangSusu@bfss.com>
Co-authored-by: Ian C <7807897+donelianc@users.noreply.github.com>
Co-authored-by: Ali Hassani <ahassanijr@gmail.com>
Co-authored-by: Raj Rajhans <me@rajrajhans.com>
Co-authored-by: Matthijs Hollemans <mail@hollance.com>
Co-authored-by: Jordan Clive <jordan.clive19@imperial.ac.uk>
Co-authored-by: jordiclive <jordiclive19@imperial.ac.uk>
Co-authored-by: Konstantin Dobler <konstantin.j.dobler@gmail.com>

* Fix nightly runs

* Fix type

* Address review comment

Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.0.1 to 9.3.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/9.0.1...9.3.0

)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Order and group topics in the right section

* Translate "Computer Vision"

* 📝 start adding inference section to task guides

* ✨ make style

* 📝 add multiple choice

* add rest of inference sections

* make style

* add compute_metric, push_to_hub, pipeline

* make style

* add updated sequence and token classification

* make style

* make edits in token classification

* add audio classification

* make style

* add asr

* make style

* add image classification

* make style

* add summarization

* make style

* add translation

* make style

* add multiple choice

* add language modeling

* add qa

* make style

* review and edits

* apply reviews

* make style

* fix call to processor

* apply audio reviews

* update to better asr model

* make style

* First draft

* Make conversion script work

* Add id2label mapping, run code quality

* Fix copies

* Add first draft of feature extractor

* Update conversion script to use feature extractor

* Make more tests pass

* Add docs

* update input_features to input_values + pad by default to max length

* Fix doc tests

* Add feature extractor tests

* Add proper padding/truncation to feature extractor

* Add support for conversion of all audioset checkpoints

* Improve docs and extend conversion script

* Fix README

* Rename spectogram to spectrogram

* Fix copies

* Add integration test

* Remove dummy conv

* Update to ast

* Update organization

* Fix init

* Rename model to AST

* Add require_torchaudio annotator

* Move import of ASTFeatureExtractor under a is_speech_available

* Fix rebase

* Add pipeline config

* Update name of classifier head

* Rename time_dimension and frequency_dimension for clarity

* Remove print statement

* Fix pipeline test

* Fix pipeline test

* Fix index table

* Fix init

* Fix conversion script

* Rename to ForAudioClassification

* Fix index table
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>