Skip to content

GitLab

Unverified Commit 2b04c209 authored by Michael Goin's avatar Michael Goin Committed by GitHub
Browse files

[Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (#14100) 


Signed-off-by: default avatarmgoin <mgoin64@gmail.com>
parent ae122b1c
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
vllm/model_executor/models/deepseek_v2.py
View file @ 2b04c209
......@@ -145,6 +145,7 @@ class DeepseekV2MoE(nn.Module):
hidden_act=config.hidden_act,
quant_config=quant_config,
reduce_results=False,
prefix=f"{prefix}.shared_experts",
)
def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment