tests/test_tokenization_fast.py · 94ff2d6ee8280c5595b92c1128c0f18e44925e56 · chenpangpang / transformers

Fast Tokenizers save pretrained should return the list of generated file paths. (#2918) · d490b5d5

Funtowicz Morgan authored Feb 20, 2020



* Correctly return the tuple of generated file(s) when calling save_pretrained
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>

* Quality and format.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>

d490b5d5

test_tokenization_fast.py 20.8 KB

Replace test_tokenization_fast.py