Skip to content

GitLab

Unverified Commit 0daeeb40 authored by Sanchit Gandhi's avatar Sanchit Gandhi Committed by GitHub
Browse files

[AutoTokenizer] Add data2vec to mapping (#25835)

parent 0e59c939
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
src/transformers/models/auto/tokenization_auto.py
View file @ 0daeeb40
...@@ -138,6 +138,7 @@ else: ...@@ -138,6 +138,7 @@ else:
), ),
("cpmant", ("CpmAntTokenizer", None)), ("cpmant", ("CpmAntTokenizer", None)),
("ctrl", ("CTRLTokenizer", None)), ("ctrl", ("CTRLTokenizer", None)),
("data2vec-audio", ("Wav2Vec2CTCTokenizer", None)),
("data2vec-text", ("RobertaTokenizer", "RobertaTokenizerFast" if is_tokenizers_available() else None)), ("data2vec-text", ("RobertaTokenizer", "RobertaTokenizerFast" if is_tokenizers_available() else None)),
("deberta", ("DebertaTokenizer", "DebertaTokenizerFast" if is_tokenizers_available() else None)), ("deberta", ("DebertaTokenizer", "DebertaTokenizerFast" if is_tokenizers_available() else None)),
( (
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment