GitLab

Moving to cleaner tokenizer version.

* update whisper logit processor

* add generate for whisper

* remove part of the whisper specific code from pipeline

* update logit processes

* major update

* enforce first timestamp

* update generate

* add more tests

* update new decoding strategy

* Apply suggestions from code review

* update docstring

* fixup

* default config will not have multilingual ar

* update expected tokenizer size, see pull on the hub for whisper-tiny

* Add doc tests

* Add OneFormer resourcesé

* Fix merge

* Fix style
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

* Fixing the pipeline with image processor.

* Update the slow test.

* Using only the first image processor.

* Include exclusion mecanism for Image processor.

* Do not handle Gitconfig, deemed as a bug.

* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Remove `conversational` changes. They are not supposed to be here.

* Address first row of comments.

* Remove OneFormer modifications.
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Add test

* Apply suggestions
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

update expected values

* Update TrainingArguments.label_names docs

* Change wording

* Change wording

* add additional kwargs handling

* fix issue when serializing

* correct order of kwargs removal for serialization in from dict

* add `dict_torch_dtype_to_str` in case a dtype is needed for generation

* add condition when adding the kwargs : not from config

* Add comment based on review
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* add test function

* default None when poping arg
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* attempts to fix:

- upcast input for `T5DenseActDense`
- add the condition `self.wo.weight.dtype != torch.int8`
- added tests on `test/mixed_int8`
- `make fixup`

* fix ci test

* fix instance segmentation post processing

* add Mask2FormerImageProcessor

use logger.info() instead of print() to emit a debug message

Return NP instead of TF tensors for our data loading pipeline

fix broken link

fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Clean all models

* Style

* Last to remove

* address review comments

* Address review comments

* added tasks from SUPPORTED_TASKS to docstrings

* make style

* sorted the tasks in the docstrtings in alphabetical order

* fix all issues with unk token

* fixup

* Replace reduce_labels with do_reduce_labels

* Replace only for __init__ and preprocess

* Add class properties with warnings

* Update tests

* use streaming dataset

* fix whisper's test

* add rescale argument to chunk_iter

* wip: adding tf example for semantic segmentation guide

* completed the working example in tf

* make style

* Update docs/source/en/tasks/semantic_segmentation.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/tasks/semantic_segmentation.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* fixed a callback doc links
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Split the examples by modality, added missing examples

* fixed a link

* Update imports and test fetcher

* Revert but keep test fetcher update

* Fix imports

* Fix all imports

* Replace fe with ip names

* Add generate kwargs to `AutomaticSpeechRecognitionPipeline` (#20952)

* Add generate kwargs to AutomaticSpeechRecognitionPipeline

* Add test for generation kwargs

* Update image processor parameters if creating with kwargs (#20866)

* Update parameters if creating with kwargs

* Shallow copy to prevent mutating input

* Pass all args in constructor dict - warnings in init

* Fix typo

* Rename tester class

* Rebase and tidy up

* Fixup

* Use ImageProcessingSavingTestMixin

* Update property ref in tests

* Update property ref in tests

* Update recently merged in models

* Small fix
Co-authored-by: bofeng huang <bofenghuang7@gmail.com>

* Replace reduce_labels with do_reduce_labels

* Replace only for __init__ and preprocess

* Update tests

* Add Japanese translation installation.mdx

* Fixed for consistency with english version

* fix ReformerForSequenceClassification doc example

* fix ReformerForMaskedLM doc example
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Optimize by not computing gradients for parameters set to requires_grad=False

* Make change to retrigger the build

* Fix isort issue

* Fix issue

* Extend conversion script

* Remove print statement
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

* Add test_image_processing_common.py

* Fix typo

* Update imports and test fetcher

* Revert but keep test fetcher update

* Fix imports

* Fix all imports

* Formatting fix

* Update tests/test_image_processing_common.py

* add converter for t5x_retrieval model

* update args

* Update src/transformers/models/t5/convert_t5x_checkpoint_to_pytorch.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* style  editing -> convert t5x to pytorch

* make style
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* First draft

* More improvements

* Add conversion script

* More improvements

* Add docs

* Address review

* Rename class to ConvEncoder

* Address review

* Apply suggestion

* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update all DETR friends

* Add corresponding test

* Improve test

* Fix bug

* Add more tests

* Set out_features to last stage by default
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix `blip` doctest

* Update src/transformers/models/blip/modeling_blip.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

skip failing test for now