Commits · 0efc4ab632ddf6f726bdf3ce1b9350f8ec183a2f · chenpangpang / transformers

07 May, 2019 1 commit
- GPT-2 option to avoid predicting special tokens · d1b6979a
  thomwolf authored May 07, 2019
  
  d1b6979a
01 May, 2019 1 commit
- gpt-2 tokenizer · db98a4a4
  thomwolf authored May 01, 2019
  
  db98a4a4
17 Apr, 2019 4 commits
- fix #497 · bcde2c61
  thomwolf authored Apr 17, 2019
  
  bcde2c61
- fix #497 · 929579f3
  thomwolf authored Apr 17, 2019
  
  929579f3
- fix GPT-2 tokenization to work also on python 3... · 5afa497c
  thomwolf authored Apr 17, 2019
  
  5afa497c
- fixed GPT-2 tokenization on python 2 · bc70779b
  thomwolf authored Apr 17, 2019
  
  bc70779b
16 Apr, 2019 1 commit
- improving GPT2 tokenization and adding tests · 18a8a15f
  thomwolf authored Apr 16, 2019
  
  18a8a15f
15 Apr, 2019 3 commits
- fixing tests · e8568a3b
  thomwolf authored Apr 15, 2019
  
  e8568a3b
- added tokenizers serialization tests · 870b734b
  thomwolf authored Apr 15, 2019
  
  870b734b
- add serialization semantics to tokenizers - fix transfo-xl tokenizer · 3e65f255
  thomwolf authored Apr 15, 2019
  
  3e65f255
06 Mar, 2019 1 commit
- fix typo - logger info · 5c85fc39
  thomwolf authored Mar 06, 2019
  
  5c85fc39
05 Mar, 2019 1 commit
- Warn instead of raising in BERT and GPT-2 tokenizers as well, to allow for pre-caching of tokens · 4a49c225
  Catalin Voss authored Mar 05, 2019
  
  4a49c225
18 Feb, 2019 3 commits
- simple · ab7f5d29
  thomwolf authored Feb 18, 2019
  
  ab7f5d29
- clean up tokenization - fix python 2 tests · b450a7fa
  thomwolf authored Feb 18, 2019
  
  b450a7fa
- update readme · d44db114
  thomwolf authored Feb 18, 2019
  
  d44db114
17 Feb, 2019 1 commit
- adding gpt2 · ffd62382
  thomwolf authored Feb 17, 2019
  
  ffd62382