GitLab

* Create PULL_REQUEST_TEMPLATE.md

Proposing to copy this neat feature from pytorch. This is a small template that let's a PR submitter tell which issue that PR closes.

* Update .github/PULL_REQUEST_TEMPLATE.md
Co-authored-by: Kevin Canwen Xu <canwenxu@126.com>
Co-authored-by: Kevin Canwen Xu <canwenxu@126.com>

* Add optuna hyperparameter search to Trainer

* @julien-c suggestions
Co-authored-by: Julien Chaumond <chaumond@gmail.com>

* Make compute_objective an arg function

* Formatting

* Rework to make it easier to add ray

* Formatting

* Initial support for Ray

* Formatting

* Polish and finalize

* Add trial id to checkpoint with Ray

* Smaller default

* Use GPU in ray if available

* Formatting

* Fix test

* Update install instruction
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Address review comments

* Formatting post-merge
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Run new isort

* More changes

* Update CI, CONTRIBUTING and benchmarks

* Fixed DataCollatorForLanguageModeling + PermutationLanguageModeling not accepting lists of lists

* Update data_collator.py

* black was grumpy

* Don't reset the type of the dataset

* Formatting

* Update trainer.py
Co-authored-by: Teven <teven.lescao@gmail.com>

* Create README.md

* Update README.md

* Create README.md

* Update README.md

* added multiple codeswitch model

* Create README.md

* Update README.md

* Create README.md

* Update README.md

Model was trained with the wrong tokenizer. Retrained with correct tokenizer - thanks for spotting @lhoestq !

I works like a charm!
Look at the output of the example code!

@julien-c

* add pegasus to docs

* Update docs/source/model_summary.rst

* added CamembertForCausalLM

* add in __init__ and auto model

* style

* doc

Signed-off-by: Morgan Funtowicz <funtowiczmo@gmail.com>

* Add a classmethod to easily build a Trainer from nlp dataset and metric

* Fix docstrings

* Split train/eval

* Formatting

* Log dropped columns + docs

* Authorize callable activations

* Poc for auto activation

* Be framework-agnostic

* Formatting

* Remove class method

* Remove unnecessary code

* Move threshold up for flaky test with Electra

* Update above as well

* xlnet fp16 bug fix

* comment cast added

* Update modeling_xlnet.py
Co-authored-by: Kevin Canwen Xu <canwenxu@126.com>

* fix distilbert

* fix typo

* TFTrainer dataset doc & fix evaluation bug

discussed in #6551

* add docstring to test/eval datasets

* Add tests to Trainer

* Test if removing long breaks everything

* Remove ugly hack

* Fix distributed test

* Use float for number of epochs

* add intro to nlp lib + links

* unique links...

* removed redundant arg in prepare_inputs

* made same change in prediction_loop

Tested in a local build of the docs.

e.g. Just above https://huggingface.co/transformers/task_summary.html#causal-language-modeling

Copy will copy the full code, e.g.

for token in top_5_tokens:
     print(sequence.replace(tokenizer.mask_token, tokenizer.decode([token])))

Instead of currently only:

for token in top_5_tokens:


>>> for token in top_5_tokens:
...     print(sequence.replace(tokenizer.mask_token, tokenizer.decode([token])))
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help reduce our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help increase our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help decrease our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help offset our carbon footprint.
Distilled models are smaller than the models they mimic. Using them instead of the large versions would help improve our carbon footprint.

Docs for the option fix:
https://sphinx-copybutton.readthedocs.io/en/latest/