GitLab

fix bug: register_for_auto_class should be defined on TFPreTrainedModel instead of TFSequenceSummary (#18607)

* Check optimizer for device on TPU

* Typo

* oob performance improvement for cpu DDP
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* add is_psutil_available check
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* Fix loss condition in DetrHungarianMatcher

* Fix costs condition in YolosHungarianMatcher

* Add LayoutLMForQuestionAnswering model

* Fix output

* Remove TF TODOs

* Add test cases

* Add docs

* TF implementation

* Fix PT/TF equivalence

* Fix loss

* make fixup

* Fix up documentation code examples

* Fix up documentation examples + test them

* Remove LayoutLMForQuestionAnswering from the auto mapping

* Docstrings

* Add better docstrings

* Undo whitespace changes

* Update tokenizers in comments

* Fixup code and remove `from_pt=True`

* Fix tests

* Revert some unexpected docstring changes

* Fix tests by overriding _prepare_for_class
Co-authored-by: Ankur Goyal <ankur@impira.com>

* Improving the documentation for "word", within the pipeline.

* Quality.

* Added Doctest for Deberta Pytorch

* Added path in documentation test file

* Added docstrings for DebertaV2

* Revert "Added docstrings for DebertaV2"

This reverts commit 307185e62a21b3bd0923444cc8a8af1747fd2600.

* Added DebertaV2 Docstrings

* groupvit to onnx

* dynamic shape for pixel values dim

* onnx conversion for owlvit

* .T to .t()

* dynamic shapes for pixel values

Co-authored-by: Esben Toke Christensen <esben.christensen@visma.com>
Co-authored-by: Lasse Reedtz <lasse.reedtz@visma.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Add dtype

* Fix Swinv2 as well
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

* add docstring for BartForCausalLM

* doc-style fic

* Fix incomplete FlaxBert outputs

* fix big_bird electra roberta

* add first generation tutorial

* VisionEnocderDecoder gradient checkpointing

* remove generation

* add tests

* Automatic detection for framework to use when exporting to ONNX

* Log message change

* Incorporating PR comments, adding unit test

* Adding tf for pip install for run_tests_onnxruntime CI

* Restoring past changes to circleci yaml and test_onnx_v2.py, tests moved to tests/onnx/test_features.py

* Fixup

* Adding test to fetcher

* Updating circleci config to log more

* Changing test class name

* Comment typo fix in tests/onnx/test_features.py
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* Moving torch_str/tf_str to self.framework_pt/tf

* Remove -rA flag in circleci config
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* Implement ONNX support for Longformer

Fix repo consistency check complaints

Fix value mismatches

Add pooler output for default model

Increase validation atol to accommodate multiple-choice error

Fix copies

Fix chunking for longer sequence lengths

Add future comment

* Fix issue in mask_invalid_locations

* Remove torch imports in configuration_longformer

* Change config access to fix LED

* Push opset version to support tril

* Work in review comments (mostly style)

* Add Longformer to ONNX tests

add warning to let the user know that the `__call__` method is faster than `encode` + `pad` for a fast tokenizer (#18693)

* add warning to let the user know that the  method is slower that  for a fast tokenizer

* user warnings

* fix layoutlmv2

* fix layout*

* change warnings into logger.warning

* fixed docstring typos

* Added missing colon
Co-authored-by: 김주영 <juyoung@zezedu.com>

* Add TFXGLM models 

* Add todo: self.supports_xla_generation = False
Co-authored-by: Daniel Stancl <stancld@Daniels-MacBook-Pro.local>
Co-authored-by: Daniel Stancl <stancld@daniels-mbp.home>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Daniel <daniel.stancl@rossum.ai>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Add minor doc-string change to include hp_name

* fix: missing type-information for kwargs

* fix: missing white-space in hyperparameter_search doc-strings

* improve add_tokens documentation

* format

and friends.

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* add first generation tutorial

* [Circle CI] Temporary fix for broken detectron2 import

* remove generation

* Fix quantization

* Save model

* Remove unused comments

* Fix formatting

* add first generation tutorial

* [LongT5 Docs] Correct docs

* correct expected string

* remove incorrect file

* Allow users to force TF availability

* Correctly name the envvar!

* Update methods to optionally rescale
This is necessary to allow for casting our images / videos to numpy arrays within the feature extractors' call. We want to do this to make sure the behaviour is as expected when flags like  are False. If some transformations aren't applied, then the output type can't be unexpected e.g. a list of PIL images instead of numpy arrays.

* Cast images to numpy arrays in call to enable consistent behaviour with different configs

* Remove accidental clip changes

* Update tests to reflect the scaling logic
We write a generic  function to handle rescaling of our arrays. In order for the API to be intuitive, we take some factor c and rescale the image values by that. This means, the rescaling done in normalize and to_numpy_array are now done with array * (1/255) instead of array / 255. This leads to small differences in the resulting image. When testing, this was in the order of 1e-8, and so deemed OK