Skip to content

GitLab

Unverified Commit 9482b0b0 authored by Michael Goin's avatar Michael Goin Committed by GitHub
Browse files

[Bugfix] Remove assertion for NVFP4 scale dynamic range (#37465) 


Signed-off-by: default avatarMichael Goin <mgoin64@gmail.com>
parent 5bc1da14
Show whitespace changes
Inline Side-by-side
Showing with 0 additions and 3 deletions
vllm/model_executor/layers/quantization/utils/marlin_utils_fp4.py
View file @ 9482b0b0
...@@ -37,9 +37,6 @@ def _nvfp4_compute_scale_factor(marlin_scales: torch.Tensor) -> float: ...@@ -37,9 +37,6 @@ def _nvfp4_compute_scale_factor(marlin_scales: torch.Tensor) -> float:
min_val = ws_float[nonzero_mask].min() min_val = ws_float[nonzero_mask].min()
if min_val < 2: if min_val < 2:
sf = (2 / min_val).log2().ceil().exp2() sf = (2 / min_val).log2().ceil().exp2()
assert (ws_float[nonzero_mask] * sf <= 448 * (2**7)).all(), (
"NVFP4 scale dynamic range too large for rescaling"
)
return sf.item() return sf.item()
return 1.0 return 1.0
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment