Commits · 58405a527bc4f52bd8c99dc330e7b37b7ce22fc6 · chenpangpang / transformers

22 Sep, 2020 1 commit
- is_pretokenized -> is_split_into_words (#7236) · 21ca1480
  Sylvain Gugger authored Sep 22, 2020
```
* is_pretokenized -> is_split_into_words

* Fix tests
```
  21ca1480
19 Aug, 2020 1 commit
- Fix #6575 (#6596) · 18ca0e91
  Sylvain Gugger authored Aug 19, 2020
  
  18ca0e91
18 Aug, 2020 1 commit
- [docs] Fix wrong newline in the middle of a paragraph (#6573) · 5a5af22e
  Romain Rigaux authored Aug 18, 2020
  
  5a5af22e
10 Aug, 2020 1 commit
- Small docfile fixes (#6328) · 6028ed92
  Sylvain Gugger authored Aug 10, 2020
  
  6028ed92
24 Jun, 2020 1 commit

Tokenization tutorial (#5257) · d12ceb48

Sylvain Gugger authored Jun 24, 2020



* All done

* Link to the tutorial

* Typo fixes
Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>

* Add metnion of the return_xxx args
Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>

d12ceb48