Unverified Commit 2c9b4cf5 authored by Lucas Wilkinson's avatar Lucas Wilkinson Committed by GitHub
Browse files

[BugFix] Fix DeepSeek-V3.1 + DeepGEMM incompatible scale shapes (#32361)


Signed-off-by: default avatarLucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: default avatarEldar Kurtić <8884008+eldarkurtic@users.noreply.github.com>
parent 9d7ae3fc
...@@ -300,6 +300,9 @@ def get_and_maybe_dequant_weights( ...@@ -300,6 +300,9 @@ def get_and_maybe_dequant_weights(
if ( if (
isinstance(layer.quant_method, Fp8LinearMethod) isinstance(layer.quant_method, Fp8LinearMethod)
and not layer.quant_method.use_marlin and not layer.quant_method.use_marlin
# DeepGEMM transforms the scales using `transform_sf_into_required_layout` into
# a layout that is not compatible with `scaled_dequantize`.
and not layer.quant_method.use_deep_gemm
): ):
weight_scales = get_attribute_fallback( weight_scales = get_attribute_fallback(
layer, ["weight_scale", "weight_scale_inv"] layer, ["weight_scale", "weight_scale_inv"]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment