Skip to content

GitLab

Unverified Commit fb22be58 authored by Mengqing Cao's avatar Mengqing Cao Committed by GitHub
Browse files

[moe][quant] add weight name case for offset (#15515) 


Signed-off-by: default avatarMengqing Cao <cmq0113@163.com>
parent 7f301dd8
Show whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
vllm/model_executor/layers/fused_moe/layer.py
View file @ fb22be58
...@@ -699,8 +699,9 @@ class FusedMoE(torch.nn.Module): ...@@ -699,8 +699,9 @@ class FusedMoE(torch.nn.Module):
tp_rank=self.tp_rank) tp_rank=self.tp_rank)
return return
# Case weight scales and zero_points # Case weight scales, zero_points and offset
if ("scale" in weight_name or "zero" in weight_name): if ("scale" in weight_name or "zero" in weight_name
or "offset" in weight_name):
# load the weight scales and zp based on the quantization scheme # load the weight scales and zp based on the quantization scheme
# supported weight scales/zp can be found in # supported weight scales/zp can be found in
# FusedMoeWeightScaleSupported # FusedMoeWeightScaleSupported
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment