tests/test_tokenization_auto.py · 676643c6d6c36927725560926a8e8f7714666d5b · chenpangpang / transformers

Sylvain Gugger authored Dec 23, 2021

* Better logic for getting tokenizer config in AutoTokenizer

* Remove needless import

* Remove debug statement

* Address review comments

676643c6

test_tokenization_auto.py 13.7 KB