[Bugfix] Fix Machete zero point issue for GPTQ models on SM90 (#21066)

Signed-off-by: mgoin <mgoin64@gmail.com>

[Bugfix] Fix Machete zero point issue for GPTQ models on SM90 (#21066)
Signed-off-by: mgoin <mgoin64@gmail.com>
28a6d542 · Michael Goin · GitHub · 58760e12 · 28a6d542
Unverified Commit 28a6d542 authored Jul 16, 2025 by Michael Goin Committed by GitHub Jul 16, 2025
Hide whitespace changes
Inline Side-by-side

Showing with 5 additions and 0 deletions

vllm/model_executor/layers/quantization/kernels/mixed_precision/machete.py ...or/layers/quantization/kernels/mixed_precision/machete.py +5 -0

No files found.
--- a/vllm/model_executor/layers/quantization/kernels/mixed_precision/machete.py
+++ b/vllm/model_executor/layers/quantization/kernels/mixed_precision/machete.py
@@ -126,6 +126,11 @@ class MacheteLinearKernel(MPLinearKernel):
        if c.has_g_idx:
            x_2d = self.act_perm(x_2d)

+        if c.zero_points:
+            assert w_zp is not None
+        else:
+            w_zp = None
+
        output = ops.machete_mm(a=x_2d,
                                b_q=w_q,
                                b_type=c.weight_type,