Skip to content

GitLab

"git@developer.sourcefind.cn:OpenDAS/dgl.git" did not exist on "5ada3fc94444244973b2ab0f9c23974c586e531d"
Unverified Commit fc1de4ff authored by eltonzheng's avatar eltonzheng Committed by GitHub
Browse files

fix transformer kernel preln config (#257)

parent 2312f04b
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
deepspeed/pt/deepspeed_cuda.py
View file @ fc1de4ff
...@@ -106,7 +106,7 @@ class DeepSpeedTransformerConfig(TransformerConfig): ...@@ -106,7 +106,7 @@ class DeepSpeedTransformerConfig(TransformerConfig):
num_hidden_layers, num_hidden_layers,
initializer_range) initializer_range)
self.fp16 = fp16 self.fp16 = fp16
self.pre_layer_norm = pre_layer_norm = True self.pre_layer_norm = pre_layer_norm
self.local_rank = local_rank self.local_rank = local_rank
self.seed = seed self.seed = seed
self.normalize_invertible = normalize_invertible self.normalize_invertible = normalize_invertible
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment