GitLab

Add `min_new_tokens` argument in generate() (implementation based on `MinNewTokensLengthLogitsProcessor`) (#21044)

add a new parameter min_new_tokens for generate()

- Remove duplicate encoder_attention_mask default value assignment

Fix docstring
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

* First draft

* More improvements

* Add convnext backbone

* Add conversion script

* Add more improvements

* Comment out to_dict

* Add to_dict method

* Add default config

* Fix config

* Fix backbone

* Fix backbone some more

* Add docs, auto mapping, tests

* Fix some tests

* Fix more tests

* Fix more tests

* Add conversion script

* Improve conversion script

* Add support for getting reshaped undownsampled hidden states

* Fix forward pass

* Add print statements

* Comment out set_shift_and_window_size

* More improvements

* Correct downsampling layers conversion

* Fix style

* First draft

* Fix conversion script

* Remove config attribute

* Fix more tests

* Update READMEs

* Update ConvNextBackbone

* Fix ConvNext tests

* Align ConvNext with Swin

* Remove files

* Fix index

* Improve docs

* Add output_attentions to model forward

* Add backbone mixin, improve tests

* More improvements

* Update init_weights

* Fix interpolation of logits

* Add UperNetImageProcessor

* Improve image processor

* Fix image processor

* Remove print statements

* Remove script

* Update import

* Add image processor tests

* Remove print statements

* Fix test

* Add integration test

* Add convnext integration test

* Update docstring

* Fix README

* Simplify config

* Apply suggestions

* Improve docs

* Rename class

* Fix test_initialization

* Fix import

* Address review

* Fix confg

* Convert all checkpoints

* Fix default backbone

* Usage same processor as segformer

* Apply suggestions

* Fix init_weights, update conversion scripts

* Improve config

* Use Auto API instead of creating a new image processor

* Fix docs

* Add doctests

* Remove ResNetConfig dependency

* Add always_partition argument

* Fix rebaseé

* Improve docs

* Convert checkpoints
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>

Fixed typo

Suppress deprecation warning

* Remove random output

* remove values

* fix copy statements

* Rework automatic code samples in docstrings

* ImageProcessor->AutoImageProcessor

* Add models to fix copies

* Last typos

* A couple more models

* Fix copies

* Add community to toctree
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* Copy English content
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* Add some translations
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* Add some translations
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* Add some translations
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* Fix position of community
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* Fix translation
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* Add translation
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* Add translation
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* Add translation
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* Add translation
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>
Signed-off-by: Shogo Hida <shogo.hida@gmail.com>

* first draft of new task summary

* make style

* review

* apply feedback

* apply feedbacks

* final touches

* typo

* change name in `__repr__`

* fix my mistake

fix torchscript tests for AltCLIP
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Fix for Past CI

* make style

* clean up

* unindent 2 blocks
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* [bnb optim] fixing test

* force 1 gpu

* fix

* fix

* fix

* finalize

* improve commentary

* fix

* cleanup

* more fixes

* Remove gradient_checkpointing from MarkupLMConfig

* Remove predict_special_tokens from OpenAIGPTConfig

* Remove enable_cls from RoCBertConfig

* Remove batch_size from TrajectoryTransformerConfig

* Remove searcher_seq_len from RealmConfig

* Remove feat_quantizer_dropout from WavLMConfig

* Remove position_biased_input from SEWDConfig

* Remove max_source_positions from Speech2Text2Config
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Fixed issue #21039 and added test for low_cpu_mem_usage

* Optimize inference only mode memory if ipex is used
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* fix code style
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

fix typo
Signed-off-by: xiaoyang zhu <zhuxiaoyang1996@gmail.com>
Signed-off-by: xiaoyang zhu <zhuxiaoyang1996@gmail.com>

Update doc for CLIPConfig

fix header level

* docs: add wandb metrics and model checkpointing to callback docstrings

* docs: update reference to wandb documentation

* fix: change default of `"WANDB_WATCH"` from ``"gradients"` to ``"false"`

* feature: add `on_save` method and update `"WANDB_LOG_MODEL` behaviour

* fix: use default wandb run names instead of `output_dir`

- removes duplicated run names from wandb workspace
- models can be logged with corresponding run names

* fix: edit deprecation warning based on review suggestions
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix: change indentation of docstrings

* fix: change indentation of docstrings and run fixup

* fix: empty commit for circleci permissions issue

* fix: format deprecation doc strings review suggestion
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* docs: Highlight WANDB_DISABLED arg in documentaion
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* fix: run fixup after updating docstrings
Co-authored-by: Bharat Ramanathan <ramanathan.parameshwaran@gohuddl.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* [Fix] Make the attention head size in distilbert an object attribute

* Fix code style
Co-authored-by: Felix Joehnk <fjoehnk@N73GCH2NDH.corp.proofpoint.com>

* small patches, forgot a line

* refactor PT

* the actual fix

remove flax file from `documentation_tests.txt`
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* start cleanup

* more updates

* more models are affected

* more updates

* update generation utils

* style

* revert change that removed reorder cachce

* update generation utils

* style

* style

* remove reorder cache

Typo fix: Corrected the word metada --> metadata

* fix levit timm conversion file

* remove set_defaults

* Add support for turning off the model uploading in ClearML

* Add documentation for the CLEARML_LOG_MODEL environment variable

* Adjust new doc addition to the new style
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Dudu Lasry <dudu.lasry@viz.ai>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

fix args passed to predict function

[NumPy] Remove references to deprecated NumPy type aliases.

This change replaces references to a number of deprecated NumPy type aliases (np.bool, np.int, np.float, np.complex, np.object, np.str) with their recommended replacement (bool, int, float, complex, object, str).

NumPy 1.24 drops the deprecated aliases, so we must remove uses before updating NumPy.
Co-authored-by: Peter Hawkins <phawkins@google.com>
Co-authored-by: Peter Hawkins <phawkins@google.com>

Added mask_time_prob and mask_time_length arguments to wav2vec2 pretraining script and readme - new branch

Fix integration test
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

* Make sure dynamic objects can be saved and reloaded

* Remove processor test