pytorch_transformers/tokenization_xlnet.py · 7424b2848f57022fcbae90cc10079935d24e6e59 · chenpangpang / transformers · GitLab

"tests/vscode:/vscode.git/clone" did not exist on "dc903e70acf9dba74d6afaa50e7b5650d6b9338a"

Find file Blame History Permalink

Merge pull request #1074 from huggingface/improved_testing · d483cd8e
Thomas Wolf authored Aug 30, 2019
```
Shortcut to special tokens' ids - fix GPT2 & RoBERTa tokenizers - improved testing for GPT/GPT-2
```
d483cd8e

tokenization_xlnet.py 8.07 KB