src/transformers/tokenization_transfo_xl.py · 79eb391586193b86c7f11bb5cf66effe0e446395 · chenpangpang / transformers · GitLab

"test/vscode:/vscode.git/clone" did not exist on "a4bf5c6ad25db291491737bad509ceca12e4a8c0"

Find file Blame History Permalink

[tokenizers] Fixing #8001 - Adding tests on tokenizers serialization (#8006) · 79eb3915
Thomas Wolf authored Oct 26, 2020
```
* fixing #8001

* make T5 tokenizer serialization more robust - style
```
79eb3915

tokenization_transfo_xl.py 30.2 KB