GitLab

* [examples] max samples can't be bigger than then len of dataset

* do tf and flax

* Split file_utils in several submodules

* Fixes

* Add back more objects

* More fixes

* Who exactly decided to import that from there?

* Second suggestion to code with code review

* Revert wront move

* Fix imports

* Adapt all imports

* Adapt all imports everywhere

* Revert this import, will fix in a separate commit

* Updates the default branch from master to main

* Links from `master` to `main`

* Typo

* Update examples/flax/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix: verify jsonl in run_translation (#14660)

* fix(run_translation.py): json/jsonl validation

Both json and jsonl are to be accepted as valid jsonlines file extension

* fix(run_translation.py): make black happy

* Ran make style

* Update run_glue.py

* Update run_glue.py

* Add model creation snippet to other scripts

* Fix style

* Add push_to_hub to no_trainer examples

* Quality

* Document integration

* Roll out to other examples

* Fix typo in deepspeed documentation

* Add missing import in deepspeed configuration

* Fix path in translation examples

* Add generate kwargs to Seq2SeqTrainingArguments

* typo

* Address review comments + doc

* Style

* Update torch.utils.data namespaces to the latest.

* Format

* Update Dataloader.

* Style

* added log_level

* fix comment

* fixed log_level

* Trigger CI

* Unfied logging

* simplified args for log_level

* main_process_first context manager

* handle multi-node, add context description

* sync desc

* bug fixes and a rename

* add extended DDP test

* set log level from CLI

* add log_level_replica + test + extended docs

* cleanup

* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* rename datasets objects to allow datasets module

* improve the doc

* style

* doc improve
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* update desc for map in all examples

* added plm

* suggestions