tests/test_tokenization_common.py · d33dc7966a8c7f04bbca7ae0ced75cbf26c38d9e · chenpangpang / transformers

"tests/vscode:/vscode.git/clone" did not exist on "a4616c6767b02c8c868bab08ccf56d59f861278d"

Improve truncation_side (#14947) · d33dc796

Nicolas Patry authored Jan 03, 2022



* Enabling `truncation_side` for Slow and Fast tokenizer.
Co-Authored-by: Niels Rogge <48327001+NielsRogge@users.noreply.github.com>

* Disable failing tests.

* Layout xlm.

* assert -> assertEqual.
Co-authored-by: Niels Rogge <48327001+NielsRogge@users.noreply.github.com>

d33dc796

test_tokenization_common.py 188 KB

Replace test_tokenization_common.py