Unverified Commit 8f1f0bf5 authored by Sören Brunk's avatar Sören Brunk Committed by GitHub
Browse files

Fix typo in LlamaTokenizerFast docstring example (#25018)

parent 3b734f50
......@@ -58,7 +58,7 @@ class LlamaTokenizerFast(PreTrainedTokenizerFast):
```
from transformers import LlamaTokenizerFast
tokenizer = LlaTokenizerFast.from_pretrained("hf-internal-testing/llama-tokenizer")
tokenizer = LlamaTokenizerFast.from_pretrained("hf-internal-testing/llama-tokenizer")
tokenizer.encode("Hello this is a test")
>>> [1, 15043, 445, 338, 263, 1243]
```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment