Skip to content

GitLab

Unverified Commit 7833b21a authored by Sylvain Gugger's avatar Sylvain Gugger Committed by GitHub
Browse files

Fix #5544 (#5551)

parent c4734840
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
src/transformers/tokenization_utils.py
View file @ 7833b21a
...@@ -649,7 +649,7 @@ class PreTrainedTokenizer(PreTrainedTokenizerBase): ...@@ -649,7 +649,7 @@ class PreTrainedTokenizer(PreTrainedTokenizerBase):
def convert_ids_to_tokens( def convert_ids_to_tokens(
self, ids: Union[int, List[int]], skip_special_tokens: bool = False self, ids: Union[int, List[int]], skip_special_tokens: bool = False
) -> Union[int, List[int]]: ) -> Union[str, List[str]]:
""" Converts a single index or a sequence of indices (integers) in a token " """ Converts a single index or a sequence of indices (integers) in a token "
(resp.) a sequence of tokens (str), using the vocabulary and added tokens. (resp.) a sequence of tokens (str), using the vocabulary and added tokens.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment