Unverified Commit 35b112d3 authored by Sadra Barikbin's avatar Sadra Barikbin Committed by GitHub
Browse files

Fix a teeny-tiny typo in `tokenization_utils_base.py`'s docstring (#31510)

Update tokenization_utils_base.py
parent 0ed3ffcb
...@@ -3645,7 +3645,7 @@ class PreTrainedTokenizerBase(SpecialTokensMixin, PushToHubMixin): ...@@ -3645,7 +3645,7 @@ class PreTrainedTokenizerBase(SpecialTokensMixin, PushToHubMixin):
and `convert_tokens_to_ids` methods. and `convert_tokens_to_ids` methods.
num_tokens_to_remove (`int`, *optional*, defaults to 0): num_tokens_to_remove (`int`, *optional*, defaults to 0):
Number of tokens to remove using the truncation strategy. Number of tokens to remove using the truncation strategy.
truncation_strategy (`str` or [`~tokenization_utils_base.TruncationStrategy`], *optional*, defaults to `False`): truncation_strategy (`str` or [`~tokenization_utils_base.TruncationStrategy`], *optional*, defaults to `'longest_first'`):
The strategy to follow for truncation. Can be: The strategy to follow for truncation. Can be:
- `'longest_first'`: Truncate to a maximum length specified with the argument `max_length` or to the - `'longest_first'`: Truncate to a maximum length specified with the argument `max_length` or to the
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment