GitLab

* Remove deprecated logic and warnings

* Add back some code that seems to be important...

* Let's just add all he nllb stuff back; removing it is a bit more involved

* Remove kwargs

* Remove more kwargs

updated examples/pytorch/language-modeling scripts and requirements.txt to require datasets>=2.14.0 (#30120)

updated requirements.txt and require_version() calls in examples/pytorch/language-modeling to require datasets>=2.14.0

fixed the broken links belogs to dataset library of transformers

Add missing spaces in adjacent strings

* docs: add space to docs

* docs: remove reduntant space

* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* pytorch examples

* tensorflow examples

* flax examples

---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Change logic

* fix bug in group_texts function, that was inserting short batches

* fully exclude short batches and return empty dict instead

* fix style

* add low_cpu_mem_usage option in run_clm.py example which will benefit LLM loading
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* update all the example and README under language-modeling
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

---------
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* Respect documentation on passive log level

* Fix test and set log level in examples

* Add doc

* Only resize embeddings when necessary

* Add comment

* zero chance anyone's using that constant no?

* `transformers-cli login` => `huggingface-cli login`

* `transformers-cli repo create` => `huggingface-cli repo create`

* `make style`

* Add examples telemetry

* Alternative approach

* Add to all other examples

* Add to templates as well

* Put framework separately

* Same for TensorFlow

* Black preview

* Fixup too!

* Fix check copies

* Use the same version as the CI

* Bump black