Commit 2a79e7d5 authored by zhangzbb's avatar zhangzbb
Browse files

Merge branch 'v0.15.1-dev_laibao' into 'v0.15.1-dev'

[BUGFIX] 修复 Qwen3-MoE Attention 中 fused RMS RoPE 的 epsilon 参数顺序错误

See merge request dcutoolkit/deeplearing/vllm!539
parents e55ba677 dec8b944
......@@ -391,9 +391,9 @@ class Qwen3MoeAttention(nn.Module):
self.rotary_emb.is_neox_style,
self.q_norm.weight,
self.k_norm.weight,
self.q_norm.variance_epsilon,
None,
None,
self.q_norm.variance_epsilon,
)
elif envs.VLLM_USE_FUSED_RMS_ROPE and positions.ndim == 2 and getattr(
self.rotary_emb, "mrope_section", None) is not None:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment