Skip to content

GitLab

Unverified Commit 8f840edd authored by Arthur's avatar Arthur Committed by GitHub
Browse files

[`Whisper`] Nit converting the tokenizer (#27349) 

* `nospeech` instead of `nocaption` for the no speech token

* oups
parent cc9f27bb
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
src/transformers/models/whisper/convert_openai_to_hf.py
View file @ 8f840edd
...@@ -235,7 +235,7 @@ def convert_tiktoken_to_hf( ...@@ -235,7 +235,7 @@ def convert_tiktoken_to_hf(
"<|transcribe|>", "<|transcribe|>",
"<|startoflm|>", "<|startoflm|>",
"<|startofprev|>", "<|startofprev|>",
"<|nocaptions|>", "<|nospeech|>",
"<|notimestamps|>", "<|notimestamps|>",
] ]
# these are special tokens, not normalized # these are special tokens, not normalized
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment