GitLab

* Update feature_extraction_clap.py

* changed all lenght to length

* Revert frozen training arguments

* TODO

feat: correct update resume update with steps

* Make training args fully immutable

* Working tests, PyTorch

* In test_trainer

* during testing

* Use proper dataclass way

* Fix test

* Another one

* Fix tf

* Lingering slow

* Exception

* Clean

* Update run_translation.py

Fixed link

* Update run_translation.py

* add missing tokens

* fix

---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Load else

* New approach

* Propagate

chorse: adding new doc on train and val

* pytorch examples

* pytorch mim no trainer

* cookiecutter

* flax examples

* missed line in pytorch run_glue

* tensorflow examples

* tensorflow run_clip

* tensorflow run_mlm

* tensorflow run_ner

* tensorflow run_clm

* pytorch example from_configs

* pytorch no trainer examples

* Revert "tensorflow run_clip"

This reverts commit 261f86ac1f1c9e05dd3fd0291e1a1f8e573781d5.

* fix: duplicated argument

* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* pytorch examples

* tensorflow examples

* flax examples

---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* make run_generation more generic for other devices

* use Accelerate to support any device type it supports.

* make style

* fix error usage of accelerator.prepare_model

* use `PartialState` to make sure everything is running on the right device

---------
Co-authored-by: statelesshz <jihuazhong1@huawei.com>

* Fix .push_to_hub and cleanup get_full_repo_name usage

* Do not rely on Python bool conversion magic

* request changes

replace `per_gpu_eval_batch_size` with `per_device_eval_batch_size` in readme of multiple-choice task (#25078)

replace `per_gpu_eval_batch_size` with `per_device_eval_batch_size`
in readme of multiple-choice

Change logic

* replace no_cuda with use_cpu in test_pytorch_examples

* remove codes that never be used

* fix style

* Add text classification example

* set the problem type and finetuning task

* ruff reformated

* fix bug for unseting label_to_id for regression

* update README.md

* fixed finetuning task

* update comment

* check if label exists in feature before removing

* add useful logging

* check if eval dataset is dict

* formatting

* Fix TypeError: Object of type int64 is not JSON serializable

* Convert numpy.float64 and numpy.int64 to float and int for json serialization

* Black reformatted examples/pytorch/token-classification/run_ner_no_trainer.py

* * make style

* Add mms ctc fine tuning

* make style

* More fixes that are needed

* make fix-copies

* make draft for README

* add new file

* move to new file

* make style

* make style

* add quick test

* make style

* make style

Fix step bugs in no trainer + load checkpoint + grad acc

Oops

Act on deprecation

* Re-enable squad test

* [all-test]

* [all-test] Fix all test command

* Fix the all-test

* Revamp test selection for the example tests

* Rename old XLA test and fake modif in run_glue

* Fixes

* Fake Trainer modif

* Remove fake modifs

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* fix bug in group_texts function, that was inserting short batches

* fully exclude short batches and return empty dict instead

* fix style

* Fix translation

skip
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* convert numpy array to list before writing to json

per_category_iou and per_category_accuracy  are ndarray in the eval_metrics

* code reformatted with make style

trainer parameters changed to save tokenizer in addition to feature_extractor