tests/test_tokenization_common.py · 519a677e87d2f72045567aed9853c020524935e0 · chenpangpang / transformers

Sylvain Gugger authored Oct 11, 2021

* Honor existing attention mask in tokenzier.pad

* Fix initialization of attention mask

* Roll the implem on all subclasses

* Fix tests

4a18337b

test_tokenization_common.py 181 KB