Skip to content

GitLab

Switch branch/tag
  1. 22 Dec, 2020 2 commits
  3. 16 Dec, 2020 1 commit
    • Patrick von Platen's avatar
      [Flax] Align FlaxBertForMaskedLM with BertForMaskedLM, implement from_pretrained, init (#9054) · 640e6fe1
      Patrick von Platen authored 
      

* save intermediate

* save intermediate

* save intermediate

* correct flax bert model file

* new module / model naming

* make style

* almost finish BERT

* finish roberta

* make fix-copies

* delete keys file

* last refactor

* fixes in run_mlm_flax.py

* remove pooled from run_mlm_flax.py`

* fix gelu | gelu_new

* remove Module from inits

* splits

* dirty print

* preventing warmup_steps == 0

* smaller splits

* make fix-copies

* dirty print

* dirty print

* initial_evaluation argument

* declaration order fix

* proper model initialization/loading

* proper initialization

* run_mlm_flax improvements: improper model inputs bugfix + automatic dataset splitting + tokenizers parallelism warning + avoiding warmup_steps=0 bug

* removed tokenizers warning hack, fixed model re-initialization

* reverted training_args.py changes

* fix flax from pretrained

* improve test in flax

* apply sylvains tips

* update init

* make 0.3.0 compatible

* revert tevens changes

* revert tevens changes 2

* finalize revert

* fix bug

* add docs

* add pretrained to init

* Update src/transformers/modeling_flax_utils.py

* fix copies

* final improvements
Co-authored-by: default avatarTevenLeScao <teven.lescao@gmail.com>
      640e6fe1
  5. 15 Dec, 2020 1 commit
  7. 11 Dec, 2020 1 commit
  9. 10 Dec, 2020 1 commit
  11. 09 Dec, 2020 1 commit
  13. 07 Dec, 2020 1 commit
  15. 23 Nov, 2020 1 commit
  17. 19 Nov, 2020 1 commit
    • Matthias's avatar
      fix small typo (#8644) · a79a96dd
      Matthias authored 
      Fixed a small typo on the XLNet and permutation language modelling section
      a79a96dd
  19. 18 Nov, 2020 2 commits
  21. 17 Nov, 2020 1 commit
  23. 12 Nov, 2020 2 commits
  25. 04 Nov, 2020 4 commits
  27. 02 Nov, 2020 1 commit
  29. 30 Oct, 2020 2 commits
  31. 29 Oct, 2020 2 commits
  33. 28 Oct, 2020 1 commit
  35. 26 Oct, 2020 1 commit
  37. 22 Oct, 2020 1 commit
  39. 12 Oct, 2020 2 commits
  41. 01 Sep, 2020 1 commit
  43. 26 Aug, 2020 1 commit
  45. 29 Jul, 2020 1 commit
  47. 07 Jul, 2020 1 commit
    • Shashank Gupta's avatar
      Added data collator for permutation (XLNet) language modeling and related calls (#5522) · 3dcb748e
      Shashank Gupta authored 
      * Added data collator for XLNet language modeling and related calls

Added DataCollatorForXLNetLanguageModeling in data/data_collator.py
to generate necessary inputs for language modeling training with
XLNetLMHeadModel. Also added related arguments, logic and calls in
examples/language-modeling/run_language_modeling.py.

Resolves: #4739, #2008 (partially)

* Changed name to `DataCollatorForPermutationLanguageModeling`

Changed the name of `DataCollatorForXLNetLanguageModeling` to the more general `DataCollatorForPermutationLanguageModelling`.
Removed the `--mlm` flag requirement for the new collator and defined a separate `--plm_probability` flag for its use.
CTRL uses a CLM loss just like GPT and GPT-2, so should work out of the box with this script (provided `past` is taken care of
similar to `mems` for XLNet).
Changed calls and imports appropriately.

* Added detailed comments, changed variable names

Added more detailed comments to `DataCollatorForPermutationLanguageModeling` in `data/data_collator.py` to explain working. Also cleaned up variable names and made them more informative.

* Added tests for new data collator

Added tests in `tests/test_trainer.py` for DataCollatorForPermutationLanguageModeling based on those in DataCollatorForLanguageModeling. A specific test has been added to check for odd-length sequences.

* Fixed styling issues
      3dcb748e
  49. 25 May, 2020 1 commit
  51. 19 May, 2020 1 commit
  53. 18 May, 2020 1 commit
  55. 15 May, 2020 1 commit
  57. 14 May, 2020 1 commit
  59. 13 May, 2020 1 commit
  61. 08 May, 2020 1 commit