Skip to content

GitLab

Commit 894812c6 authored by Julien Chaumond's avatar Julien Chaumond
Browse files

Fixup mapping

parent b20f11d4
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
src/transformers/tokenization_auto.py
View file @ 894812c6
......@@ -60,8 +60,8 @@ TOKENIZER_MAPPING = OrderedDict(
(DistilBertConfig, DistilBertTokenizer),
(AlbertConfig, AlbertTokenizer),
(CamembertConfig, CamembertTokenizer),
(RobertaConfig, XLMRobertaTokenizer),
(XLMRobertaConfig, RobertaTokenizer),
(RobertaConfig, RobertaTokenizer),
(XLMRobertaConfig, XLMRobertaTokenizer),
(BertConfig, BertTokenizer),
(OpenAIGPTConfig, OpenAIGPTTokenizer),
(GPT2Config, GPT2Tokenizer),
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment