Unverified Commit c8d3a402 authored by Binyao Jiang's avatar Binyao Jiang Committed by GitHub
Browse files

Bug: apply final_hidden_states*=self.routed_scaling_factor at MoE lay… (#8511)


Co-authored-by: default avatarCheng Wan <54331508+ch-wan@users.noreply.github.com>
parent 7e831efe
......@@ -304,7 +304,7 @@ class EPMoE(FusedMoE):
m_max * self.start_expert_id,
BLOCK_SIZE=512,
)
return output
return output * self.routed_scaling_factor
class DeepEPMoE(EPMoE):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment