Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
6e3b0144
"...git@developer.sourcefind.cn:chenpangpang/transformers.git" did not exist on "10b76987fc45a36aab5c5549dd607e2475c32e73"
Unverified
Commit
6e3b0144
authored
Nov 15, 2022
by
TilmannR
Committed by
GitHub
Nov 15, 2022
Browse files
Fix docstring of CLIPTokenizer(Fast) (#20233)
parent
cf7b98b8
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
2 additions
and
2 deletions
+2
-2
src/transformers/models/clip/tokenization_clip.py
src/transformers/models/clip/tokenization_clip.py
+1
-1
src/transformers/models/clip/tokenization_clip_fast.py
src/transformers/models/clip/tokenization_clip_fast.py
+1
-1
No files found.
src/transformers/models/clip/tokenization_clip.py
View file @
6e3b0144
...
@@ -275,7 +275,7 @@ class CLIPTokenizer(PreTrainedTokenizer):
...
@@ -275,7 +275,7 @@ class CLIPTokenizer(PreTrainedTokenizer):
unk_token (`str`, *optional*, defaults to `<|endoftext|>`):
unk_token (`str`, *optional*, defaults to `<|endoftext|>`):
The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
token instead.
token instead.
bos_token (`str`, *optional*, defaults to `<|
end
oftext|>`):
bos_token (`str`, *optional*, defaults to `<|
start
oftext|>`):
The beginning of sequence token.
The beginning of sequence token.
eos_token (`str`, *optional*, defaults to `<|endoftext|>`):
eos_token (`str`, *optional*, defaults to `<|endoftext|>`):
The end of sequence token.
The end of sequence token.
...
...
src/transformers/models/clip/tokenization_clip_fast.py
View file @
6e3b0144
...
@@ -63,7 +63,7 @@ class CLIPTokenizerFast(PreTrainedTokenizerFast):
...
@@ -63,7 +63,7 @@ class CLIPTokenizerFast(PreTrainedTokenizerFast):
unk_token (`str`, *optional*, defaults to `<|endoftext|>`):
unk_token (`str`, *optional*, defaults to `<|endoftext|>`):
The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
token instead.
token instead.
bos_token (`str`, *optional*, defaults to `<|
end
oftext|>`):
bos_token (`str`, *optional*, defaults to `<|
start
oftext|>`):
The beginning of sequence token.
The beginning of sequence token.
eos_token (`str`, *optional*, defaults to `<|endoftext|>`):
eos_token (`str`, *optional*, defaults to `<|endoftext|>`):
The end of sequence token.
The end of sequence token.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment