Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
17292440
"docs/vscode:/vscode.git/clone" did not exist on "b6b79faa7ea92f67bbbac621caebeb1ecc774108"
Unverified
Commit
17292440
authored
Dec 30, 2022
by
Samuel Xu
Committed by
GitHub
Dec 30, 2022
Browse files
Fixing DistilBert error message (#20945)
Fixing error message
parent
881fa716
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
src/transformers/models/distilbert/tokenization_distilbert.py
...transformers/models/distilbert/tokenization_distilbert.py
+1
-1
No files found.
src/transformers/models/distilbert/tokenization_distilbert.py
View file @
17292440
...
@@ -166,7 +166,7 @@ class DistilBertTokenizer(PreTrainedTokenizer):
...
@@ -166,7 +166,7 @@ class DistilBertTokenizer(PreTrainedTokenizer):
if
not
os
.
path
.
isfile
(
vocab_file
):
if
not
os
.
path
.
isfile
(
vocab_file
):
raise
ValueError
(
raise
ValueError
(
f
"Can't find a vocabulary file at path '
{
vocab_file
}
'. To load the vocabulary from a Google pretrained"
f
"Can't find a vocabulary file at path '
{
vocab_file
}
'. To load the vocabulary from a Google pretrained"
" model use `tokenizer = BertTokenizer.from_pretrained(PRETRAINED_MODEL_NAME)`"
" model use `tokenizer =
Distil
BertTokenizer.from_pretrained(PRETRAINED_MODEL_NAME)`"
)
)
self
.
vocab
=
load_vocab
(
vocab_file
)
self
.
vocab
=
load_vocab
(
vocab_file
)
self
.
ids_to_tokens
=
collections
.
OrderedDict
([(
ids
,
tok
)
for
tok
,
ids
in
self
.
vocab
.
items
()])
self
.
ids_to_tokens
=
collections
.
OrderedDict
([(
ids
,
tok
)
for
tok
,
ids
in
self
.
vocab
.
items
()])
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment