Commits · 88111de07c40797aaca619be693616c3c4cda4bd · chenpangpang / transformers

30 Aug, 2019 4 commits
- saving and reloading tokenizer configurations · 88111de0
  thomwolf authored Aug 30, 2019
  
  88111de0
- adding xlm 17 and 100 models and config on aws · 3871b8a1
  thomwolf authored Aug 30, 2019
  
  3871b8a1
- adding 17 and 100 xlm models · 8678ff8d
  thomwolf authored Aug 30, 2019
  
  8678ff8d
- Added option to setup pretrained tokenizer arguments · 82462c5c
  thomwolf authored Aug 30, 2019
  
  82462c5c
28 Aug, 2019 1 commit
- Match order of casing in OSS XLM; Improve document; Clean up dependency · ca4baf8c
  Shijie Wu authored Aug 27, 2019
  
  ca4baf8c
27 Aug, 2019 1 commit
- add kwargs to base encode function · a175a9dc
  thomwolf authored Aug 27, 2019
  
  a175a9dc
24 Aug, 2019 2 commits
- Add use_lang_emb to config · f1b01874
  Shijie Wu authored Aug 23, 2019
  
  f1b01874
- Add custom tokenizer for zh and ja · e85123d3
  Shijie Wu authored Aug 23, 2019
  
  e85123d3
23 Aug, 2019 1 commit

Tokenization behave the same as original XLM proprocessing for most languages... · 436ce072

Shijie Wu authored Aug 23, 2019

Tokenization behave the same as original XLM proprocessing for most languages except zh, ja and th; Change API to allow specifying language in `tokenize`

436ce072

22 Aug, 2019 1 commit
- reraise EnvironmentError in from_pretrained functions of Model and Tokenizer · c603d099
  Abhishek Rao authored Aug 22, 2019
  
  c603d099
21 Aug, 2019 4 commits
- Fix at config rather than model · 14eef67e
  Abhishek Rao authored Aug 21, 2019
  
  14eef67e
- reraise exception · 296df2b1
  Abhishek Rao authored Aug 21, 2019
  
  296df2b1
- Add max length · fdc487d8
  thomwolf authored Aug 21, 2019
  
  fdc487d8
- adding gpt-2 large · aa05dc89
  thomwolf authored Aug 21, 2019
  
  aa05dc89
20 Aug, 2019 11 commits
- adding proxies options for the from_pretrained methods · 43489756
  thomwolf authored Aug 20, 2019
  
  43489756
- Update tokenization_xlm.py · 388e3251
  Guillem García Subies authored Aug 20, 2019
  
  388e3251
- Update tokenization_openai.py · f5e2ed0f
  Guillem García Subies authored Aug 20, 2019
  
  f5e2ed0f
- Update tokenization_openai.py · 562b9983
  Guillem García Subies authored Aug 20, 2019
  
  562b9983
- Update tokenization_openai.py · bb044462
  Guillem García Subies authored Aug 20, 2019
  
  bb044462
- Update tokenization_xlm.py · bfd75056
  Guillem García Subies authored Aug 20, 2019
  
  bfd75056
- fix #1034 · 6d0aa739
  thomwolf authored Aug 20, 2019
  
  6d0aa739
- minor typo · b0b9b809
  Julien Chaumond authored Aug 20, 2019
  
  b0b9b809
- fix #808 · 53c8f700
  thomwolf authored Aug 20, 2019
  
  53c8f700
- fix #1014 · 901dde0e
  thomwolf authored Aug 20, 2019
  
  901dde0e
- add force_download option to from_pretrained methods · fecaed0e
  thomwolf authored Aug 20, 2019
  
  fecaed0e
19 Aug, 2019 1 commit
- Doc: loading from config alone does not load the model weights · c589862b
  Lysandre authored Aug 19, 2019
  
  c589862b
16 Aug, 2019 1 commit
- Order of strings in AutoModel/AutoTokenizer updated. · ab052806
  LysandreJik authored Aug 16, 2019
  
  ab052806
15 Aug, 2019 4 commits
- Added RoBERTa tokenizer to AutoTokenizer · 83dba0b6
  LysandreJik authored Aug 15, 2019
  
  83dba0b6
- Added RoBERTa to AutoModel/AutoConfig · e24e19ce
  LysandreJik authored Aug 15, 2019
  
  e24e19ce
- Release: 1.1.0 · fe02e45e
  LysandreJik authored Aug 15, 2019
  
  fe02e45e
- Warning for RoBERTa sequences encoded without special tokens. · 83081701
  LysandreJik authored Aug 15, 2019
  
  83081701
14 Aug, 2019 2 commits
- Doc · 572dcfd1
  LysandreJik authored Aug 14, 2019
  
  572dcfd1
- Re-implemented tokenize() iteratively in PreTrainedTokenizer. · 9ce36e3e
  samvelyan authored Aug 14, 2019
  
  9ce36e3e
13 Aug, 2019 2 commits
- Added special tokens <pad> and <mask> to RoBERTa. · 39f426be
  LysandreJik authored Aug 13, 2019
  
  39f426be
- Fixed error with encoding · 3d87991f
  LysandreJik authored Aug 13, 2019
  
  3d87991f
12 Aug, 2019 3 commits
- Added integration tests for sequence builders. · 634a3172
  LysandreJik authored Aug 12, 2019
  
  634a3172
- Added documentation and changed parameters for special_tokens_sentences_pair. · 22ac004a
  LysandreJik authored Aug 12, 2019
  
  22ac004a
- Fixup 9d060314 · b3d83d68
  Julien Chaumond authored Aug 12, 2019
  
  b3d83d68
10 Aug, 2019 1 commit
- fix #993 · c683c3d5
  thomwolf authored Aug 10, 2019
  
  c683c3d5
09 Aug, 2019 1 commit
- Corrected logger.error info · 70607664
  Kevin Trebing authored Aug 09, 2019
```
Signed-off-by: Kevin Trebing <Kevin.Trebing@gmx.net>
```
  70607664