tests/test_tokenization_auto.py · d33dc7966a8c7f04bbca7ae0ced75cbf26c38d9e · chenpangpang / transformers

Improve truncation_side (#14947) · d33dc796

Nicolas Patry authored Jan 03, 2022



* Enabling `truncation_side` for Slow and Fast tokenizer.
Co-Authored-by: Niels Rogge <48327001+NielsRogge@users.noreply.github.com>

* Disable failing tests.

* Layout xlm.

* assert -> assertEqual.
Co-authored-by: Niels Rogge <48327001+NielsRogge@users.noreply.github.com>

d33dc796

test_tokenization_auto.py 13.8 KB

Replace test_tokenization_auto.py