GitLab

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* dd parameter --config_overrides for run_mlm_wwm.py

* linter

* fix

* Apply suggestions from code review
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

The emoji version must be either 0.5.4 or 0.6.0. Newer emoji versions have been updated to newer versions of the Emoji Charts, thus not consistent with the one used for pre-processing the pre-training Tweet corpus (i.e. not consistent with the vocab).

* Add -e flag

* add check

* create new keys

* run python setup.py build install

* add comments

* change to develop
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Fix doc notebooks links

* Remove missing section

* Fixup all examples

* Fix multiple deletions of the same files in save_pretrained

* Add is_main_process argument

BertModelForSequenceClassification -> BertForSequenceClassification

* Fix `distributed_concat` with scalar tensor

* Update trainer_pt_utils.py

* Add fix

* Apply suggestion from code review
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>

1. Fixes evaluation errors popping up when you train/eval on squad v2 (one was newly encountered and one that was previously reported Running SQuAD 1.0 sample command raises IndexError #15401 but not completely fixed).
2. Removes boolean arguments that don't use store_true. Please, don't use these: *ANY non-empty string is being converted to True in this case and this clearly is not the desired behavior (and it creates a LOT of confusion).
3. All no-trainer test scripts are now saving metric values in the same way (with the right prefix eval_), which is consistent with the trainer-based versions.
4. Adds forgotten model.eval() in the no-trainer versions. This improved some results, but not everything (see the discussion in the end). Please, see the F1 scores and the discussion below.

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* Add first draft

* Improve script and README

* Improve README

* Apply suggestions from code review

* Improve script, add link to resulting model

* Add corresponding test

* Adjust learning rate

* Avoid repeated per-lang filtering

* Language groups and logits preprocessing

* Style

* add roformer onnx config

FIx Iterations for decoder

* apply torch int div

* black linting fixup

* update path to torch_int_div

* clarify imports

* Limit the use of PreTrainedModel.device

* Fix

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* added deit onnx config

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* XLA min len, forced eos, and forced bos
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* add torch.cuda.empty_cache in some PT RAG tests

* torch.cuda.empty_cache in tearDownModule()

* tearDown()

* add gc.collect()
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>