Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
a99f7f5c
Unverified
Commit
a99f7f5c
authored
Apr 12, 2021
by
cronoik
Committed by
GitHub
Apr 12, 2021
Browse files
Minor typos fixed (#11182)
parent
26212c14
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
3 deletions
+3
-3
src/transformers/models/reformer/configuration_reformer.py
src/transformers/models/reformer/configuration_reformer.py
+3
-3
No files found.
src/transformers/models/reformer/configuration_reformer.py
View file @
a99f7f5c
...
...
@@ -52,7 +52,7 @@ class ReformerConfig(PretrainedConfig):
The standard deviation of the normal_initializer for initializing the weight matrices of the axial
positional encodings.
axial_pos_shape (:obj:`List[int]`, `optional`, defaults to :obj:`[64, 64]`):
The position dims of the axial position encodings. During training the product of the position dims has to
The position dims of the axial position encodings. During training
,
the product of the position dims has to
be equal to the sequence length.
For more information on how axial position embeddings work, see `Axial Position Encodings
...
...
@@ -88,7 +88,7 @@ class ReformerConfig(PretrainedConfig):
initializer_range (:obj:`float`, `optional`, defaults to 0.02):
The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
is_decoder (:obj:`bool`, `optional`, defaults to :obj:`False`):
Whether o
t
not to use a causal mask in addition to the :obj:`attention_mask` passed to
Whether o
r
not to use a causal mask in addition to the :obj:`attention_mask` passed to
:class:`~transformers.ReformerModel`. When using the Reformer for causal language modeling, this argument
should be set to :obj:`True`.
layer_norm_eps (:obj:`float`, `optional`, defaults to 1e-12):
...
...
@@ -134,7 +134,7 @@ class ReformerConfig(PretrainedConfig):
pad_token_id (:obj:`int`, `optional`, defaults to 0):
The token id for the padding token.
vocab_size (:obj:`int`, `optional`, defaults to 320):\
Vocabulary size of the
BERT
model. Defines the number of different tokens that can be represented by the
Vocabulary size of the
Reformer
model. Defines the number of different tokens that can be represented by the
:obj:`inputs_ids` passed when calling :class:`~transformers.ReformerModel`.
tie_word_embeddings (:obj:`bool`, `optional`, defaults to :obj:`False`):
Whether to tie input and output embeddings.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment