tests/test_tokenization_xlm_roberta.py · d6afbd323d2b3ffdc139127b78b8caa672e985a1 · chenpangpang / transformers

XLM-R Tokenizer now passes common tests + Integration tests (#3198) · d6afbd32

Lysandre Debut authored Mar 18, 2020

* XLM-R now passes common tests + Integration tests

* Correct mask index

* Model input names

* Style

* Remove text preprocessing

* Unneccessary import

d6afbd32

test_tokenization_xlm_roberta.py 6.26 KB

Replace test_tokenization_xlm_roberta.py