Unverified Commit f7ab9554 authored by Jimmy's avatar Jimmy Committed by GitHub
Browse files

fix(glm45): disable reduce scatter (#11665)


Co-authored-by: default avatarShangming Cai <csmthu@gmail.com>
parent dbbd4e18
......@@ -656,7 +656,7 @@ class Glm4MoeDecoderLayer(DeepseekV2DecoderLayer):
layer_scatter_modes=self.layer_scatter_modes,
input_layernorm=self.input_layernorm,
post_attention_layernorm=self.post_attention_layernorm,
allow_reduce_scatter=True,
allow_reduce_scatter=False,
)
def forward(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment