Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
faf25c04
Unverified
Commit
faf25c04
authored
Jul 25, 2023
by
Kashif Rasul
Committed by
GitHub
Jul 25, 2023
Browse files
[Docs] fix rope_scaling doc string (#25072)
fix rope_scaling doc string
parent
c0742b15
Changes
3
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
3 additions
and
3 deletions
+3
-3
src/transformers/models/deprecated/open_llama/configuration_open_llama.py
.../models/deprecated/open_llama/configuration_open_llama.py
+1
-1
src/transformers/models/gpt_neox/configuration_gpt_neox.py
src/transformers/models/gpt_neox/configuration_gpt_neox.py
+1
-1
src/transformers/models/llama/configuration_llama.py
src/transformers/models/llama/configuration_llama.py
+1
-1
No files found.
src/transformers/models/deprecated/open_llama/configuration_open_llama.py
View file @
faf25c04
...
@@ -68,7 +68,7 @@ class OpenLlamaConfig(PretrainedConfig):
...
@@ -68,7 +68,7 @@ class OpenLlamaConfig(PretrainedConfig):
tie_word_embeddings(`bool`, *optional*, defaults to `False`):
tie_word_embeddings(`bool`, *optional*, defaults to `False`):
Whether to tie weight embeddings
Whether to tie weight embeddings
rope_scaling (`Dict`, *optional*):
rope_scaling (`Dict`, *optional*):
Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports t
hree
scaling
Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports t
wo
scaling
strategies: linear and dynamic. Their scaling factor must be an float greater than 1. The expected format
strategies: linear and dynamic. Their scaling factor must be an float greater than 1. The expected format
is `{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update
is `{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update
`max_position_embeddings` to the expected new maximum. See the following thread for more information on how
`max_position_embeddings` to the expected new maximum. See the following thread for more information on how
...
...
src/transformers/models/gpt_neox/configuration_gpt_neox.py
View file @
faf25c04
...
@@ -79,7 +79,7 @@ class GPTNeoXConfig(PretrainedConfig):
...
@@ -79,7 +79,7 @@ class GPTNeoXConfig(PretrainedConfig):
Whether to use a "parallel" formulation in each Transformer layer, which can provide a slight training
Whether to use a "parallel" formulation in each Transformer layer, which can provide a slight training
speedup at large scales (e.g. 20B).
speedup at large scales (e.g. 20B).
rope_scaling (`Dict`, *optional*):
rope_scaling (`Dict`, *optional*):
Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports t
hree
scaling
Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports t
wo
scaling
strategies: linear and dynamic. Their scaling factor must be an float greater than 1. The expected format
strategies: linear and dynamic. Their scaling factor must be an float greater than 1. The expected format
is `{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update
is `{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update
`max_position_embeddings` to the expected new maximum. See the following thread for more information on how
`max_position_embeddings` to the expected new maximum. See the following thread for more information on how
...
...
src/transformers/models/llama/configuration_llama.py
View file @
faf25c04
...
@@ -78,7 +78,7 @@ class LlamaConfig(PretrainedConfig):
...
@@ -78,7 +78,7 @@ class LlamaConfig(PretrainedConfig):
tie_word_embeddings(`bool`, *optional*, defaults to `False`):
tie_word_embeddings(`bool`, *optional*, defaults to `False`):
Whether to tie weight embeddings
Whether to tie weight embeddings
rope_scaling (`Dict`, *optional*):
rope_scaling (`Dict`, *optional*):
Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports t
hree
scaling
Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports t
wo
scaling
strategies: linear and dynamic. Their scaling factor must be an float greater than 1. The expected format
strategies: linear and dynamic. Their scaling factor must be an float greater than 1. The expected format
is `{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update
is `{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update
`max_position_embeddings` to the expected new maximum. See the following thread for more information on how
`max_position_embeddings` to the expected new maximum. See the following thread for more information on how
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment