Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
7777db15
Unverified
Commit
7777db15
authored
Dec 24, 2020
by
Bram Vanroy
Committed by
GitHub
Dec 24, 2020
Browse files
Update tokenization_utils_base.py (#9293)
Missing "s" typo
parent
71963a66
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
src/transformers/tokenization_utils_base.py
src/transformers/tokenization_utils_base.py
+1
-1
No files found.
src/transformers/tokenization_utils_base.py
View file @
7777db15
...
...
@@ -3179,7 +3179,7 @@ class PreTrainedTokenizerBase(SpecialTokensMixin):
assert
already_has_special_tokens
and
token_ids_1
is
None
,
(
"You cannot use ``already_has_special_tokens=False`` with this tokenizer. "
"Please use a slow (full python) tokenizer to activate this argument."
"Or set `return_special_token_mask=True` when calling the encoding method "
"Or set `return_special_token
s
_mask=True` when calling the encoding method "
"to get the special tokens mask in any tokenizer. "
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment