Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
088c1880
Unverified
Commit
088c1880
authored
Mar 25, 2022
by
Sylvain Gugger
Committed by
GitHub
Mar 25, 2022
Browse files
Big file_utils cleanup (#16396)
* Big file_utils cleanup * This one still needs to be treated separately
parent
2b23e080
Changes
222
Show whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
41 additions
and
41 deletions
+41
-41
src/transformers/models/ibert/modeling_ibert.py
src/transformers/models/ibert/modeling_ibert.py
+1
-1
src/transformers/models/imagegpt/feature_extraction_imagegpt.py
...ansformers/models/imagegpt/feature_extraction_imagegpt.py
+1
-1
src/transformers/models/imagegpt/modeling_imagegpt.py
src/transformers/models/imagegpt/modeling_imagegpt.py
+1
-1
src/transformers/models/layoutlm/modeling_layoutlm.py
src/transformers/models/layoutlm/modeling_layoutlm.py
+1
-1
src/transformers/models/layoutlm/modeling_tf_layoutlm.py
src/transformers/models/layoutlm/modeling_tf_layoutlm.py
+1
-1
src/transformers/models/layoutlmv2/feature_extraction_layoutlmv2.py
...ormers/models/layoutlmv2/feature_extraction_layoutlmv2.py
+1
-1
src/transformers/models/layoutlmv2/modeling_layoutlmv2.py
src/transformers/models/layoutlmv2/modeling_layoutlmv2.py
+1
-1
src/transformers/models/layoutlmv2/tokenization_layoutlmv2.py
...transformers/models/layoutlmv2/tokenization_layoutlmv2.py
+2
-2
src/transformers/models/led/modeling_led.py
src/transformers/models/led/modeling_led.py
+3
-3
src/transformers/models/led/modeling_tf_led.py
src/transformers/models/led/modeling_tf_led.py
+4
-4
src/transformers/models/longformer/modeling_longformer.py
src/transformers/models/longformer/modeling_longformer.py
+1
-1
src/transformers/models/longformer/modeling_tf_longformer.py
src/transformers/models/longformer/modeling_tf_longformer.py
+2
-2
src/transformers/models/luke/modeling_luke.py
src/transformers/models/luke/modeling_luke.py
+1
-1
src/transformers/models/luke/tokenization_luke.py
src/transformers/models/luke/tokenization_luke.py
+2
-2
src/transformers/models/lxmert/modeling_lxmert.py
src/transformers/models/lxmert/modeling_lxmert.py
+1
-1
src/transformers/models/lxmert/modeling_tf_lxmert.py
src/transformers/models/lxmert/modeling_tf_lxmert.py
+2
-2
src/transformers/models/m2m_100/modeling_m2m_100.py
src/transformers/models/m2m_100/modeling_m2m_100.py
+3
-3
src/transformers/models/marian/modeling_flax_marian.py
src/transformers/models/marian/modeling_flax_marian.py
+3
-3
src/transformers/models/marian/modeling_marian.py
src/transformers/models/marian/modeling_marian.py
+4
-4
src/transformers/models/marian/modeling_tf_marian.py
src/transformers/models/marian/modeling_tf_marian.py
+6
-6
No files found.
src/transformers/models/ibert/modeling_ibert.py
View file @
088c1880
...
...
@@ -722,7 +722,7 @@ IBERT_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
src/transformers/models/imagegpt/feature_extraction_imagegpt.py
View file @
088c1880
...
...
@@ -117,7 +117,7 @@ class ImageGPTFeatureExtractor(FeatureExtractionMixin, ImageFeatureExtractionMix
tensor. In case of a NumPy array/PyTorch tensor, each image should be of shape (C, H, W), where C is a
number of channels, H and W are image height and width.
return_tensors (`str` or [`~
file_
utils.TensorType`], *optional*, defaults to `'np'`):
return_tensors (`str` or [`~utils.TensorType`], *optional*, defaults to `'np'`):
If set, will return tensors of a particular framework. Acceptable values are:
- `'tf'`: Return TensorFlow `tf.constant` objects.
...
...
src/transformers/models/imagegpt/modeling_imagegpt.py
View file @
088c1880
...
...
@@ -608,7 +608,7 @@ IMAGEGPT_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
src/transformers/models/layoutlm/modeling_layoutlm.py
View file @
088c1880
...
...
@@ -695,7 +695,7 @@ LAYOUTLM_INPUTS_DOCSTRING = r"""
If set to `True`, the hidden states of all layers are returned. See `hidden_states` under returned tensors
for more detail.
return_dict (`bool`, *optional*):
If set to `True`, the model will return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
If set to `True`, the model will return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
src/transformers/models/layoutlm/modeling_tf_layoutlm.py
View file @
088c1880
...
...
@@ -891,7 +891,7 @@ LAYOUTLM_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
training (`bool`, *optional*, defaults to `False`):
Whether or not to use the model in training mode (some modules like dropout modules have different
behaviors between training and evaluation).
...
...
src/transformers/models/layoutlmv2/feature_extraction_layoutlmv2.py
View file @
088c1880
...
...
@@ -131,7 +131,7 @@ class LayoutLMv2FeatureExtractor(FeatureExtractionMixin, ImageFeatureExtractionM
The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch
tensor. In case of a NumPy array/PyTorch tensor, each image should be of shape (C, H, W), where C is a
number of channels, H and W are image height and width.
return_tensors (`str` or [`~
file_
utils.TensorType`], *optional*, defaults to `'np'`):
return_tensors (`str` or [`~utils.TensorType`], *optional*, defaults to `'np'`):
If set, will return tensors of a particular framework. Acceptable values are:
- `'tf'`: Return TensorFlow `tf.constant` objects.
...
...
src/transformers/models/layoutlmv2/modeling_layoutlmv2.py
View file @
088c1880
...
...
@@ -682,7 +682,7 @@ LAYOUTLMV2_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
src/transformers/models/layoutlmv2/tokenization_layoutlmv2.py
View file @
088c1880
...
...
@@ -60,7 +60,7 @@ PRETRAINED_INIT_CONFIGURATION = {
LAYOUTLMV2_ENCODE_PLUS_ADDITIONAL_KWARGS_DOCSTRING
=
r
"""
add_special_tokens (`bool`, *optional*, defaults to `True`):
Whether or not to encode the sequences with the special tokens relative to their model.
padding (`bool`, `str` or [`~
file_
utils.PaddingStrategy`], *optional*, defaults to `False`):
padding (`bool`, `str` or [`~utils.PaddingStrategy`], *optional*, defaults to `False`):
Activates and controls padding. Accepts the following values:
- `True` or `'longest'`: Pad to the longest sequence in the batch (or no padding if only a single
...
...
@@ -97,7 +97,7 @@ LAYOUTLMV2_ENCODE_PLUS_ADDITIONAL_KWARGS_DOCSTRING = r"""
pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value. This is especially useful to enable
the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta).
return_tensors (`str` or [`~
file_
utils.TensorType`], *optional*):
return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are:
- `'tf'`: Return TensorFlow `tf.constant` objects.
...
...
src/transformers/models/led/modeling_led.py
View file @
088c1880
...
...
@@ -1590,7 +1590,7 @@ LED_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
@@ -1748,7 +1748,7 @@ class LEDEncoder(LEDPreTrainedModel):
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
for more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
output_attentions
=
output_attentions
if
output_attentions
is
not
None
else
self
.
config
.
output_attentions
output_hidden_states
=
(
...
...
@@ -2003,7 +2003,7 @@ class LEDDecoder(LEDPreTrainedModel):
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
for more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
output_attentions
=
output_attentions
if
output_attentions
is
not
None
else
self
.
config
.
output_attentions
output_hidden_states
=
(
...
...
src/transformers/models/led/modeling_tf_led.py
View file @
088c1880
...
...
@@ -1600,8 +1600,8 @@ LED_INPUTS_DOCSTRING = r"""
more detail. This argument can be used only in eager mode, in graph mode the value in the config will be
used instead.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple. This argument can be used
in
eager mode, in graph mode the value will always be set to True.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. This argument can be used
in
eager mode, in graph mode the value will always be set to True.
training (`bool`, *optional*, defaults to `False`):
Whether or not to use the model in training mode (some modules like dropout modules have different
behaviors between training and evaluation).
...
...
@@ -1701,7 +1701,7 @@ class TFLEDEncoder(tf.keras.layers.Layer):
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
for more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
inputs
=
input_processing
(
func
=
self
.
call
,
...
...
@@ -1983,7 +1983,7 @@ class TFLEDDecoder(tf.keras.layers.Layer):
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
for more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
inputs
=
input_processing
(
func
=
self
.
call
,
...
...
src/transformers/models/longformer/modeling_longformer.py
View file @
088c1880
...
...
@@ -1486,7 +1486,7 @@ LONGFORMER_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
src/transformers/models/longformer/modeling_tf_longformer.py
View file @
088c1880
...
...
@@ -1974,8 +1974,8 @@ LONGFORMER_INPUTS_DOCSTRING = r"""
more detail. This argument can be used only in eager mode, in graph mode the value in the config will be
used instead.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple. This argument can be used
in
eager mode, in graph mode the value will always be set to True.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. This argument can be used
in
eager mode, in graph mode the value will always be set to True.
training (`bool`, *optional*, defaults to `False`):
Whether or not to use the model in training mode (some modules like dropout modules have different
behaviors between training and evaluation).
...
...
src/transformers/models/luke/modeling_luke.py
View file @
088c1880
...
...
@@ -868,7 +868,7 @@ LUKE_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
src/transformers/models/luke/tokenization_luke.py
View file @
088c1880
...
...
@@ -1108,7 +1108,7 @@ class LukeTokenizer(RobertaTokenizer):
List[int]]]*) so you can use this method during preprocessing as well as in a PyTorch Dataloader
collate function. Instead of `List[int]` you can have tensors (numpy arrays, PyTorch tensors or
TensorFlow tensors), see the note above for the return type.
padding (`bool`, `str` or [`~
file_
utils.PaddingStrategy`], *optional*, defaults to `True`):
padding (`bool`, `str` or [`~utils.PaddingStrategy`], *optional*, defaults to `True`):
Select a strategy to pad the returned sequences (according to the model's padding side and padding
index) among:
...
...
@@ -1129,7 +1129,7 @@ class LukeTokenizer(RobertaTokenizer):
Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific tokenizer's default, defined by the `return_outputs` attribute. [What are attention
masks?](../glossary#attention-mask)
return_tensors (`str` or [`~
file_
utils.TensorType`], *optional*):
return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are:
- `'tf'`: Return TensorFlow `tf.constant` objects.
...
...
src/transformers/models/lxmert/modeling_lxmert.py
View file @
088c1880
...
...
@@ -875,7 +875,7 @@ LXMERT_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
src/transformers/models/lxmert/modeling_tf_lxmert.py
View file @
088c1880
...
...
@@ -929,8 +929,8 @@ LXMERT_INPUTS_DOCSTRING = r"""
more detail. This argument can be used only in eager mode, in graph mode the value in the config will be
used instead.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple. This argument can be used
in
eager mode, in graph mode the value will always be set to True.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. This argument can be used
in
eager mode, in graph mode the value will always be set to True.
training (`bool`, *optional*, defaults to `False`):
Whether or not to use the model in training mode (some modules like dropout modules have different
behaviors between training and evaluation).
...
...
src/transformers/models/m2m_100/modeling_m2m_100.py
View file @
088c1880
...
...
@@ -667,7 +667,7 @@ M2M_100_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
@@ -753,7 +753,7 @@ class M2M100Encoder(M2M100PreTrainedModel):
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
for more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
output_attentions
=
output_attentions
if
output_attentions
is
not
None
else
self
.
config
.
output_attentions
output_hidden_states
=
(
...
...
@@ -952,7 +952,7 @@ class M2M100Decoder(M2M100PreTrainedModel):
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
for more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
output_attentions
=
output_attentions
if
output_attentions
is
not
None
else
self
.
config
.
output_attentions
output_hidden_states
=
(
...
...
src/transformers/models/marian/modeling_flax_marian.py
View file @
088c1880
...
...
@@ -136,7 +136,7 @@ MARIAN_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
@@ -167,7 +167,7 @@ MARIAN_ENCODE_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
MARIAN_DECODE_INPUTS_DOCSTRING
=
r
"""
...
...
@@ -213,7 +213,7 @@ MARIAN_DECODE_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
src/transformers/models/marian/modeling_marian.py
View file @
088c1880
...
...
@@ -634,7 +634,7 @@ MARIAN_INPUTS_DOCSTRING = r"""
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for
more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
...
...
@@ -723,7 +723,7 @@ class MarianEncoder(MarianPreTrainedModel):
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
for more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
output_attentions
=
output_attentions
if
output_attentions
is
not
None
else
self
.
config
.
output_attentions
output_hidden_states
=
(
...
...
@@ -942,7 +942,7 @@ class MarianDecoder(MarianPreTrainedModel):
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
for more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
"""
output_attentions
=
output_attentions
if
output_attentions
is
not
None
else
self
.
config
.
output_attentions
output_hidden_states
=
(
...
...
@@ -1643,7 +1643,7 @@ class MarianForCausalLM(MarianPreTrainedModel):
Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors
for more detail.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple.
Returns:
...
...
src/transformers/models/marian/modeling_tf_marian.py
View file @
088c1880
...
...
@@ -654,8 +654,8 @@ MARIAN_INPUTS_DOCSTRING = r"""
more detail. This argument can be used only in eager mode, in graph mode the value in the config will be
used instead.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple. This argument can be used
in
eager mode, in graph mode the value will always be set to True.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. This argument can be used
in
eager mode, in graph mode the value will always be set to True.
training (`bool`, *optional*, defaults to `False`):
Whether or not to use the model in training mode (some modules like dropout modules have different
behaviors between training and evaluation).
...
...
@@ -745,8 +745,8 @@ class TFMarianEncoder(tf.keras.layers.Layer):
for more detail. This argument can be used only in eager mode, in graph mode the value in the config
will be used instead.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple. This argument can be
used
in eager mode, in graph mode the value will always be set to True.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. This argument can be
used
in eager mode, in graph mode the value will always be set to True.
training (`bool`, *optional*, defaults to `False`):
Whether or not to use the model in training mode (some modules like dropout modules have different
behaviors between training and evaluation).
...
...
@@ -928,8 +928,8 @@ class TFMarianDecoder(tf.keras.layers.Layer):
for more detail. This argument can be used only in eager mode, in graph mode the value in the config
will be used instead.
return_dict (`bool`, *optional*):
Whether or not to return a [`~
file_
utils.ModelOutput`] instead of a plain tuple. This argument can be
used
in eager mode, in graph mode the value will always be set to True.
Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. This argument can be
used
in eager mode, in graph mode the value will always be set to True.
training (`bool`, *optional*, defaults to `False`):
Whether or not to use the model in training mode (some modules like dropout modules have different
behaviors between training and evaluation).
...
...
Prev
1
2
3
4
5
6
7
8
9
10
…
12
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment