tests/test_tokenization_auto.py · 80af1048cf34d4cff54c13c99d3abfd4e9b3f4eb · chenpangpang / transformers

Improve truncation_side (#14947) · d33dc796

Nicolas Patry authored Jan 03, 2022



* Enabling `truncation_side` for Slow and Fast tokenizer.
Co-Authored-by: Niels Rogge <48327001+NielsRogge@users.noreply.github.com>

* Disable failing tests.

* Layout xlm.

* assert -> assertEqual.
Co-authored-by: Niels Rogge <48327001+NielsRogge@users.noreply.github.com>

d33dc796

test_tokenization_auto.py 13.8 KB

Replace test_tokenization_auto.py