src/transformers/models/llama/tokenization_llama.py · aaccf1844eccbb90cc923378e3c37a6b143d03fb · chenpangpang / transformers

[docstring] Fix docstring for `LlamaTokenizer` and `LlamaTokenizerFast` (#26669) · aaccf184

Minho Ryang authored Oct 12, 2023

* [docstring] Fix docstring for `LlamaTokenizer` and `LlamaTokenizerFast`

* [docstring] Fix docstring typo at `LlamaTokenizer` and `LlamaTokenizerFast`

aaccf184

tokenization_llama.py 20.9 KB

Replace tokenization_llama.py