Unverified Commit 9b468a7c authored by Matthijs Hollemans's avatar Matthijs Hollemans Committed by GitHub
Browse files

workaround documentation rendering bug (#21189)

parent 464c86ac
...@@ -107,7 +107,7 @@ class SequenceFeatureExtractor(FeatureExtractionMixin): ...@@ -107,7 +107,7 @@ class SequenceFeatureExtractor(FeatureExtractionMixin):
If set will pad the sequence to a multiple of the provided value. If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
>= 7.5 (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. `>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*): return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor's default. to the specific feature_extractor's default.
...@@ -250,7 +250,7 @@ class SequenceFeatureExtractor(FeatureExtractionMixin): ...@@ -250,7 +250,7 @@ class SequenceFeatureExtractor(FeatureExtractionMixin):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. `>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
...@@ -309,7 +309,7 @@ class SequenceFeatureExtractor(FeatureExtractionMixin): ...@@ -309,7 +309,7 @@ class SequenceFeatureExtractor(FeatureExtractionMixin):
max_length: maximum length of the returned list and optionally padding length (see below) max_length: maximum length of the returned list and optionally padding length (see below)
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. `>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
truncation: truncation:
(optional) Activates truncation to cut input sequences longer than `max_length` to `max_length`. (optional) Activates truncation to cut input sequences longer than `max_length` to `max_length`.
""" """
......
...@@ -100,7 +100,7 @@ LAYOUTLMV2_ENCODE_KWARGS_DOCSTRING = r""" ...@@ -100,7 +100,7 @@ LAYOUTLMV2_ENCODE_KWARGS_DOCSTRING = r"""
argument defines the number of overlapping tokens. argument defines the number of overlapping tokens.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_tensors (`str` or [`~file_utils.TensorType`], *optional*): return_tensors (`str` or [`~file_utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are: If set, will return tensors instead of list of python integers. Acceptable values are:
...@@ -1288,7 +1288,7 @@ class LayoutLMv2Tokenizer(PreTrainedTokenizer): ...@@ -1288,7 +1288,7 @@ class LayoutLMv2Tokenizer(PreTrainedTokenizer):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -710,7 +710,7 @@ class LayoutLMv2TokenizerFast(PreTrainedTokenizerFast): ...@@ -710,7 +710,7 @@ class LayoutLMv2TokenizerFast(PreTrainedTokenizerFast):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -97,7 +97,7 @@ LAYOUTLMV3_ENCODE_KWARGS_DOCSTRING = r""" ...@@ -97,7 +97,7 @@ LAYOUTLMV3_ENCODE_KWARGS_DOCSTRING = r"""
argument defines the number of overlapping tokens. argument defines the number of overlapping tokens.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_tensors (`str` or [`~file_utils.TensorType`], *optional*): return_tensors (`str` or [`~file_utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are: If set, will return tensors instead of list of python integers. Acceptable values are:
...@@ -146,7 +146,7 @@ LAYOUTLMV3_ENCODE_PLUS_ADDITIONAL_KWARGS_DOCSTRING = r""" ...@@ -146,7 +146,7 @@ LAYOUTLMV3_ENCODE_PLUS_ADDITIONAL_KWARGS_DOCSTRING = r"""
argument defines the number of overlapping tokens. argument defines the number of overlapping tokens.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_tensors (`str` or [`~utils.TensorType`], *optional*): return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are: If set, will return tensors instead of list of python integers. Acceptable values are:
...@@ -1420,7 +1420,7 @@ class LayoutLMv3Tokenizer(PreTrainedTokenizer): ...@@ -1420,7 +1420,7 @@ class LayoutLMv3Tokenizer(PreTrainedTokenizer):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -762,7 +762,7 @@ class LayoutLMv3TokenizerFast(PreTrainedTokenizerFast): ...@@ -762,7 +762,7 @@ class LayoutLMv3TokenizerFast(PreTrainedTokenizerFast):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -82,7 +82,7 @@ LAYOUTXLM_ENCODE_KWARGS_DOCSTRING = r""" ...@@ -82,7 +82,7 @@ LAYOUTXLM_ENCODE_KWARGS_DOCSTRING = r"""
argument defines the number of overlapping tokens. argument defines the number of overlapping tokens.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_tensors (`str` or [`~file_utils.TensorType`], *optional*): return_tensors (`str` or [`~file_utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are: If set, will return tensors instead of list of python integers. Acceptable values are:
...@@ -1118,7 +1118,7 @@ class LayoutXLMTokenizer(PreTrainedTokenizer): ...@@ -1118,7 +1118,7 @@ class LayoutXLMTokenizer(PreTrainedTokenizer):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -85,7 +85,7 @@ LAYOUTXLM_ENCODE_KWARGS_DOCSTRING = r""" ...@@ -85,7 +85,7 @@ LAYOUTXLM_ENCODE_KWARGS_DOCSTRING = r"""
argument defines the number of overlapping tokens. argument defines the number of overlapping tokens.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_tensors (`str` or [`~file_utils.TensorType`], *optional*): return_tensors (`str` or [`~file_utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are: If set, will return tensors instead of list of python integers. Acceptable values are:
...@@ -674,7 +674,7 @@ class LayoutXLMTokenizerFast(PreTrainedTokenizerFast): ...@@ -674,7 +674,7 @@ class LayoutXLMTokenizerFast(PreTrainedTokenizerFast):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -1440,7 +1440,7 @@ class LukeTokenizer(PreTrainedTokenizer): ...@@ -1440,7 +1440,7 @@ class LukeTokenizer(PreTrainedTokenizer):
The maximum length of the entity sequence. The maximum length of the entity sequence.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_attention_mask (`bool`, *optional*): return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific tokenizer's default, defined by the `return_outputs` attribute. [What are attention to the specific tokenizer's default, defined by the `return_outputs` attribute. [What are attention
...@@ -1584,7 +1584,7 @@ class LukeTokenizer(PreTrainedTokenizer): ...@@ -1584,7 +1584,7 @@ class LukeTokenizer(PreTrainedTokenizer):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -96,7 +96,7 @@ MARKUPLM_ENCODE_PLUS_ADDITIONAL_KWARGS_DOCSTRING = r""" ...@@ -96,7 +96,7 @@ MARKUPLM_ENCODE_PLUS_ADDITIONAL_KWARGS_DOCSTRING = r"""
argument defines the number of overlapping tokens. argument defines the number of overlapping tokens.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_tensors (`str` or [`~file_utils.TensorType`], *optional*): return_tensors (`str` or [`~file_utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are: If set, will return tensors instead of list of python integers. Acceptable values are:
...@@ -1392,7 +1392,7 @@ class MarkupLMTokenizer(PreTrainedTokenizer): ...@@ -1392,7 +1392,7 @@ class MarkupLMTokenizer(PreTrainedTokenizer):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -806,7 +806,7 @@ class MarkupLMTokenizerFast(PreTrainedTokenizerFast): ...@@ -806,7 +806,7 @@ class MarkupLMTokenizerFast(PreTrainedTokenizerFast):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -275,7 +275,7 @@ class MCTCTFeatureExtractor(SequenceFeatureExtractor): ...@@ -275,7 +275,7 @@ class MCTCTFeatureExtractor(SequenceFeatureExtractor):
If set will pad the sequence to a multiple of the provided value. If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
>= 7.5 (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. `>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*): return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor's default. to the specific feature_extractor's default.
......
...@@ -1238,7 +1238,7 @@ class MLukeTokenizer(PreTrainedTokenizer): ...@@ -1238,7 +1238,7 @@ class MLukeTokenizer(PreTrainedTokenizer):
The maximum length of the entity sequence. The maximum length of the entity sequence.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_attention_mask (`bool`, *optional*): return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific tokenizer's default, defined by the `return_outputs` attribute. [What are attention to the specific tokenizer's default, defined by the `return_outputs` attribute. [What are attention
...@@ -1383,7 +1383,7 @@ class MLukeTokenizer(PreTrainedTokenizer): ...@@ -1383,7 +1383,7 @@ class MLukeTokenizer(PreTrainedTokenizer):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -160,7 +160,7 @@ class Speech2TextFeatureExtractor(SequenceFeatureExtractor): ...@@ -160,7 +160,7 @@ class Speech2TextFeatureExtractor(SequenceFeatureExtractor):
If set will pad the sequence to a multiple of the provided value. If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
>= 7.5 (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. `>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*): return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor's default. to the specific feature_extractor's default.
......
...@@ -223,7 +223,7 @@ TAPAS_ENCODE_PLUS_ADDITIONAL_KWARGS_DOCSTRING = r""" ...@@ -223,7 +223,7 @@ TAPAS_ENCODE_PLUS_ADDITIONAL_KWARGS_DOCSTRING = r"""
which it will tokenize. This is useful for NER or token classification. which it will tokenize. This is useful for NER or token classification.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_tensors (`str` or [`~utils.TensorType`], *optional*): return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are: If set, will return tensors instead of list of python integers. Acceptable values are:
...@@ -1852,7 +1852,7 @@ class TapasTokenizer(PreTrainedTokenizer): ...@@ -1852,7 +1852,7 @@ class TapasTokenizer(PreTrainedTokenizer):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -106,7 +106,7 @@ TAPEX_ENCODE_PLUS_ADDITIONAL_KWARGS_DOCSTRING = r""" ...@@ -106,7 +106,7 @@ TAPEX_ENCODE_PLUS_ADDITIONAL_KWARGS_DOCSTRING = r"""
argument defines the number of overlapping tokens. argument defines the number of overlapping tokens.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_tensors (`str` or [`~file_utils.TensorType`], *optional*): return_tensors (`str` or [`~file_utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are: If set, will return tensors instead of list of python integers. Acceptable values are:
......
...@@ -136,7 +136,7 @@ class Wav2Vec2FeatureExtractor(SequenceFeatureExtractor): ...@@ -136,7 +136,7 @@ class Wav2Vec2FeatureExtractor(SequenceFeatureExtractor):
If set will pad the sequence to a multiple of the provided value. If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
>= 7.5 (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. `>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*): return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor's default. to the specific feature_extractor's default.
......
...@@ -88,7 +88,7 @@ WAV2VEC2_KWARGS_DOCSTRING = r""" ...@@ -88,7 +88,7 @@ WAV2VEC2_KWARGS_DOCSTRING = r"""
length (like XLNet) truncation/padding to a maximum length will be deactivated. length (like XLNet) truncation/padding to a maximum length will be deactivated.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_tensors (`str` or [`~utils.TensorType`], *optional*): return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are: If set, will return tensors instead of list of python integers. Acceptable values are:
......
...@@ -240,7 +240,7 @@ class WhisperFeatureExtractor(SequenceFeatureExtractor): ...@@ -240,7 +240,7 @@ class WhisperFeatureExtractor(SequenceFeatureExtractor):
If set will pad the sequence to a multiple of the provided value. If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
>= 7.5 (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. `>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*): return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor's default. to the specific feature_extractor's default.
......
...@@ -1343,7 +1343,7 @@ ENCODE_KWARGS_DOCSTRING = r""" ...@@ -1343,7 +1343,7 @@ ENCODE_KWARGS_DOCSTRING = r"""
which it will tokenize. This is useful for NER or token classification. which it will tokenize. This is useful for NER or token classification.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
return_tensors (`str` or [`~utils.TensorType`], *optional*): return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are: If set, will return tensors instead of list of python integers. Acceptable values are:
...@@ -2902,7 +2902,7 @@ class PreTrainedTokenizerBase(SpecialTokensMixin, PushToHubMixin): ...@@ -2902,7 +2902,7 @@ class PreTrainedTokenizerBase(SpecialTokensMixin, PushToHubMixin):
If set will pad the sequence to a multiple of the provided value. If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask (`bool`, *optional*): return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific tokenizer's default, defined by the `return_outputs` attribute. to the specific tokenizer's default, defined by the `return_outputs` attribute.
...@@ -3339,7 +3339,7 @@ class PreTrainedTokenizerBase(SpecialTokensMixin, PushToHubMixin): ...@@ -3339,7 +3339,7 @@ class PreTrainedTokenizerBase(SpecialTokensMixin, PushToHubMixin):
- 'right': pads on the right of the sequences - 'right': pads on the right of the sequences
pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value. pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
>= 7.5 (Volta). `>= 7.5` (Volta).
return_attention_mask: return_attention_mask:
(optional) Set to False to avoid returning attention mask (default: set to model specifics) (optional) Set to False to avoid returning attention mask (default: set to model specifics)
""" """
......
...@@ -346,7 +346,7 @@ class PreTrainedTokenizerFast(PreTrainedTokenizerBase): ...@@ -346,7 +346,7 @@ class PreTrainedTokenizerFast(PreTrainedTokenizerBase):
The stride to use when handling overflow. The stride to use when handling overflow.
pad_to_multiple_of (`int`, *optional*): pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta). the use of Tensor Cores on NVIDIA hardware with compute capability `>= 7.5` (Volta).
""" """
_truncation = self._tokenizer.truncation _truncation = self._tokenizer.truncation
_padding = self._tokenizer.padding _padding = self._tokenizer.padding
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment