Skip to content

GitLab

Commit 2036eb73 authored by zhuwenwen's avatar zhuwenwen
Browse files

Merge branch 'v0.15.1-dev_yql_3.3' into 'v0.15.1-dev' 

修复awq-marlin的bug

See merge request dcutoolkit/deeplearing/vllm!454
parents aaf8c95f 09bdce60
Show whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
vllm/_custom_ops.py
View file @ 2036eb73
...@@ -1515,6 +1515,7 @@ def awq_marlin_moe_repack( ...@@ -1515,6 +1515,7 @@ def awq_marlin_moe_repack(
output = torch.empty((num_experts, size_k // 16, size_n * (num_bits // 2)), output = torch.empty((num_experts, size_k // 16, size_n * (num_bits // 2)),
device=b_q_weight.device, device=b_q_weight.device,
dtype=b_q_weight.dtype) dtype=b_q_weight.dtype)
for e in range(num_experts):
output[e] = op.awq_marlin_repack(b_q_weight[e], size_k, output[e] = op.awq_marlin_repack(b_q_weight[e], size_k,
size_n, num_bits) size_n, num_bits)
return output return output
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment