Skip to content

GitLab

"...git@developer.sourcefind.cn:2222/OpenDAS/vllm_cscc.git" did not exist on "d46d490c275091b4900ce10aa21032c222e85180"
Commit e92bb9ea authored by gaoqiong's avatar gaoqiong
Browse files

修改增加SlimQuantW4A8Int8MoEMethod 获取intermediate_size_per_partition 支持

parent 8b1e4ef0
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/model_executor/layers/fused_moe/layer.py
View file @ e92bb9ea
......@@ -821,7 +821,7 @@ class FusedMoE(torch.nn.Module):
if (self.quant_method.__class__.__name__ in ("BlockInt8MoEMethod")):
moe_quant_params["intermediate_size"] = self.intermediate_size_per_partition
if (self.quant_method.__class__.__name__ in ("W8A8Int8MoEMethod")):
if (self.quant_method.__class__.__name__ in ("SlimQuantW4A8Int8MoEMethod")):
moe_quant_params["intermediate_size"] = self.intermediate_size_per_partition
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment