tests/models/perceiver/test_tokenization_perceiver.py · 986526a0e4f5ab803581074e9e4069c3edcff1dc · chenpangpang / transformers

"lightx2v/deploy/vscode:/vscode.git/clone" did not exist on "bcb74974151bc3b4ffbab073f7b71647403362d9"

Replace `as_target` context managers by direct calls (#18325) · 986526a0

Sylvain Gugger authored Jul 29, 2022



* Preliminary work on tokenizers

* Quality + fix tests

* Treat processors

* Fix pad

* Remove all uses of  in tests, docs and examples

* Replace all as_target_tokenizer

* Fix tests

* Fix quality

* Update examples/flax/image-captioning/run_image_captioning_flax.py
Co-authored-by: amyeroberts <amy@huggingface.co>

* Style
Co-authored-by: amyeroberts <amy@huggingface.co>

986526a0

test_tokenization_perceiver.py 13.4 KB

Replace test_tokenization_perceiver.py