Skip to content

GitLab

Unverified Commit 03416ead authored by haoyangli-amd's avatar haoyangli-amd Committed by GitHub
Browse files

[bugfix][quantization] Fix fp8 per_tensor scale shape (#30257) 


Signed-off-by: default avatarHaoyang Li <lihaoyang0109@gmail.com>
parent c72ea107
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/_custom_ops.py
View file @ 03416ead
...@@ -1726,7 +1726,7 @@ def scaled_fp8_quant( ...@@ -1726,7 +1726,7 @@ def scaled_fp8_quant(
output, input, scale, scale_ub output, input, scale, scale_ub
) )
else: else:
scale = torch.empty((1, 1), device=input.device, dtype=torch.float32) scale = torch.empty(1, device=input.device, dtype=torch.float32)
torch.ops._C.dynamic_scaled_fp8_quant(output, input, scale) torch.ops._C.dynamic_scaled_fp8_quant(output, input, scale)
else: else:
assert scale.numel() == 1, f"{scale.shape}" assert scale.numel() == 1, f"{scale.shape}"
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment