tests/t5/test_tokenization_t5.py · 31616b8d613dcb7ac69b562d51b42d0db379f72f · chenpangpang / transformers

[T5 Tokenizer] Model has no fixed position ids - there is no hardcode… (#16990) · 31616b8d

Patrick von Platen authored May 02, 2022



* [T5 Tokenizer] Model has no fixed position ids - there is no hardcoded max length

* [T5 Tokenizer] Model has no fixed position ids - there is no hardcoded max length

* correct t5 tokenizer

* correct t5 tokenizer

* fix test

* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* finish
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

31616b8d

test_tokenization_t5.py 18 KB

Replace test_tokenization_t5.py