Skip to content

GitLab

Switch branch/tag
  1. 07 Jul, 2020 3 commits
    • Abel's avatar
      Make T5 compatible with ONNX (#5518) · 69122657
      Abel authored 
      

* Default decoder inputs to encoder ones for T5 if neither are specified.

* Fixing typo, now all tests are passing.

* Changing einsum to operations supported by onnx

* Adding a test to ensure T5 can be exported to onnx op>9

* Modified test for onnx export to make it faster

* Styling changes.

* Styling changes.

* Changing notation for matrix multiplication
Co-authored-by: default avatarAbel Riboulot <tkai@protomail.com>
      69122657
    • Patrick von Platen's avatar
      [Reformer] Adapt Reformer MaskedLM Attn mask (#5560) · 989ae326
      Patrick von Platen authored 
      * fix attention mask

* fix slow test

* refactor attn masks

* fix fp16 generate test
      989ae326
    • Shashank Gupta's avatar
      Added data collator for permutation (XLNet) language modeling and related calls (#5522) · 3dcb748e
      Shashank Gupta authored 
      * Added data collator for XLNet language modeling and related calls

Added DataCollatorForXLNetLanguageModeling in data/data_collator.py
to generate necessary inputs for language modeling training with
XLNetLMHeadModel. Also added related arguments, logic and calls in
examples/language-modeling/run_language_modeling.py.

Resolves: #4739, #2008 (partially)

* Changed name to `DataCollatorForPermutationLanguageModeling`

Changed the name of `DataCollatorForXLNetLanguageModeling` to the more general `DataCollatorForPermutationLanguageModelling`.
Removed the `--mlm` flag requirement for the new collator and defined a separate `--plm_probability` flag for its use.
CTRL uses a CLM loss just like GPT and GPT-2, so should work out of the box with this script (provided `past` is taken care of
similar to `mems` for XLNet).
Changed calls and imports appropriately.

* Added detailed comments, changed variable names

Added more detailed comments to `DataCollatorForPermutationLanguageModeling` in `data/data_collator.py` to explain working. Also cleaned up variable names and made them more informative.

* Added tests for new data collator

Added tests in `tests/test_trainer.py` for DataCollatorForPermutationLanguageModeling based on those in DataCollatorForLanguageModeling. A specific test has been added to check for odd-length sequences.

* Fixed styling issues
      3dcb748e
  3. 06 Jul, 2020 1 commit
    • Anthony MOI's avatar
      Various tokenizers fixes (#5558) · 5787e4c1
      Anthony MOI authored 
      * BertTokenizerFast - Do not specify strip_accents by default

* Bump tokenizers to new version

* Add test for AddedToken serialization
      5787e4c1
  5. 03 Jul, 2020 2 commits
  7. 02 Jul, 2020 1 commit
  9. 01 Jul, 2020 8 commits
  11. 30 Jun, 2020 1 commit
  13. 29 Jun, 2020 1 commit
  15. 28 Jun, 2020 1 commit
  17. 26 Jun, 2020 4 commits
  19. 25 Jun, 2020 3 commits
  21. 24 Jun, 2020 4 commits
  23. 23 Jun, 2020 4 commits
  25. 22 Jun, 2020 4 commits
    • Thomas Wolf's avatar
      [tokenizers] Fix #5081 and improve backward compatibility (#5125) · ebc36108
      Thomas Wolf authored 
      * fix #5081 and improve backward compatibility (slightly)

* add nlp to setup.cfg - style and quality

* align default to previous default

* remove test that doesn't generalize
      ebc36108
    • Joseph Liu's avatar
      Output hidden states (#4978) · f4e1f022
      Joseph Liu authored 
      

* Configure all models to use output_hidden_states as argument passed to foward()

* Pass all tests

* Remove cast_bool_to_primitive in TF Flaubert model

* correct tf xlnet

* add pytorch test

* add tf test

* Fix broken tests

* Configure all models to use output_hidden_states as argument passed to foward()

* Pass all tests

* Remove cast_bool_to_primitive in TF Flaubert model

* correct tf xlnet

* add pytorch test

* add tf test

* Fix broken tests

* Refactor output_hidden_states for mobilebert

* Reset and remerge to master
Co-authored-by: default avatarJoseph Liu <joseph.liu@coinflex.com>
Co-authored-by: default avatarpatrickvonplaten <patrick.v.platen@gmail.com>
      f4e1f022
    • RafaelWO's avatar
      Added feature to move added tokens in vocabulary for Transformer-XL (#4953) · b99ad457
      RafaelWO authored 
      

* Fixed resize_token_embeddings for transfo_xl model

* Fixed resize_token_embeddings for transfo_xl.

Added custom methods to TransfoXLPreTrainedModel for resizing layers of
the AdaptiveEmbedding.

* Updated docstring

* Fixed resizinhg cutoffs; added check for new size of embedding layer.

* Added test for resize_token_embeddings

* Fixed code quality

* Fixed unchanged cutoffs in model.config

* Added feature to move added tokens in tokenizer.

* Fixed code quality

* Added feature to move added tokens in tokenizer.

* Fixed code quality

* Fixed docstring, renamed sym to 	oken.
Co-authored-by: default avatarRafael Weingartner <rweingartner.its-b2015@fh-salzburg.ac.at>
      b99ad457
    • Patrick von Platen's avatar
      Benchmarks (#4912) · fa0be6d7
      Patrick von Platen authored 
      * finish benchmark

* fix isort

* fix setup cfg

* retab

* fix time measuring of tf graph mode

* fix tf cuda

* clean code

* better error message
      fa0be6d7
  27. 19 Jun, 2020 2 commits
  29. 18 Jun, 2020 1 commit