Unverified Commit 930a2414 authored by Wentao Ye's avatar Wentao Ye Committed by GitHub
Browse files

[Bug] R1 Accuracy: Fix `routed_scaling_factor` Double Mul Issue (#24119)


Signed-off-by: default avataryewentao256 <zhyanwentao@126.com>
parent 457e4719
......@@ -160,7 +160,8 @@ class DeepseekV2MoE(nn.Module):
topk_group=config.topk_group,
prefix=f"{prefix}.experts",
scoring_func=config.scoring_func,
routed_scaling_factor=self.routed_scaling_factor,
# we do scaling outside, set factor to 1.0 to avoid double mul
routed_scaling_factor=1.0,
e_score_correction_bias=self.gate.e_score_correction_bias,
enable_eplb=self.enable_eplb,
num_redundant_experts=self.n_redundant_experts)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment