Skip to content

GitLab

Unverified Commit 8f1f0bf5 authored by Sören Brunk's avatar Sören Brunk Committed by GitHub
Browse files

Fix typo in LlamaTokenizerFast docstring example (#25018)

parent 3b734f50
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
src/transformers/models/llama/tokenization_llama_fast.py
View file @ 8f1f0bf5
...@@ -58,7 +58,7 @@ class LlamaTokenizerFast(PreTrainedTokenizerFast): ...@@ -58,7 +58,7 @@ class LlamaTokenizerFast(PreTrainedTokenizerFast):
``` ```
from transformers import LlamaTokenizerFast from transformers import LlamaTokenizerFast
tokenizer = LlaTokenizerFast.from_pretrained("hf-internal-testing/llama-tokenizer") tokenizer = LlamaTokenizerFast.from_pretrained("hf-internal-testing/llama-tokenizer")
tokenizer.encode("Hello this is a test") tokenizer.encode("Hello this is a test")
>>> [1, 15043, 445, 338, 263, 1243] >>> [1, 15043, 445, 338, 263, 1243]
``` ```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment