Unverified Commit 56a62c31 authored by Matthias Gehre's avatar Matthias Gehre Committed by GitHub
Browse files

[Bugfix] Reject channelwise quantization (group_size <= 0) in ExllamaLinearKernel (#37331)


Signed-off-by: default avatarMatthias Gehre <matthias.gehre@amd.com>
parent 1779c098
......@@ -59,6 +59,13 @@ class ExllamaLinearKernel(MPLinearKernel):
f"{cls.SUPPORTED_QUANT_TYPES}",
)
if c.group_size <= 0:
return (
False,
f"Group size ({c.group_size}) must be positive, "
"Exllama does not support channelwise quantization",
)
if c.full_weight_shape[0] % c.group_size != 0:
return (
False,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment