tests/models/llama/test_tokenization_llama.py · 9a5d468ba0562e2d5edf9da787881fa227132bca · chenpangpang / transformers

Arthur authored May 30, 2023

* Update the processor when changing add_eos and add_bos

* fixup

* update

* add a test

* fix failing tests

* fixup

6fc0454b

test_tokenization_llama.py 21.1 KB