Skip to content

GitLab

Unverified Commit d3c18c9c authored by Yuxuan Zhang's avatar Yuxuan Zhang Committed by GitHub
Browse files

fuse fp32 for GLM-4.5 e_score_correction_bias (#22143) 


Signed-off-by: default avatarzRzRzRzRzRzRzR <2448370773@qq.com>
parent 83f7bbb3
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 3 deletions
vllm/model_executor/models/glm4_moe.py
View file @ d3c18c9c
......@@ -125,9 +125,8 @@ class Glm4MoE(nn.Module):
quant_config=None,
prefix=f"{prefix}.gate")
# noaux_tc is not set in transformers new config now
self.gate.e_score_correction_bias = (nn.Parameter(
torch.empty(config.n_routed_experts)))
self.gate.e_score_correction_bias = nn.Parameter(
torch.empty(config.n_routed_experts, dtype=torch.float32))
# Load balancing settings.
vllm_config = get_current_vllm_config()
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment