tests/models/udop/test_tokenization_udop.py · ecfe9be7054e81f8841b8e97e6599e1a2d35ed7e · chenpangpang / transformers

[UDOP] Add special tokens to tokenizer (#29594) · ecfe9be7

NielsRogge authored Apr 19, 2024

* Add special tokens

* Add special tokens

* Use fmt

* Uncomment code

* Add test

* Remove scripts

* Address comments

* Improve tests

* Address comment

* Remove flag

ecfe9be7

test_tokenization_udop.py 95 KB

Replace test_tokenization_udop.py