Skip to content

GitLab

Unverified Commit f952bbc8 authored by Michael Goin's avatar Michael Goin Committed by GitHub
Browse files

[Misc] Fix input_scale typing in w8a8_utils.py (#6579)

parent 9364f74e
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
vllm/model_executor/layers/quantization/utils/w8a8_utils.py
View file @ f952bbc8
...@@ -104,7 +104,7 @@ def apply_fp8_linear( ...@@ -104,7 +104,7 @@ def apply_fp8_linear(
input: torch.Tensor, input: torch.Tensor,
weight: torch.Tensor, weight: torch.Tensor,
weight_scale: torch.Tensor, weight_scale: torch.Tensor,
input_scale: torch.Tensor, input_scale: Optional[torch.Tensor] = None,
input_scale_ub: Optional[torch.Tensor] = None, input_scale_ub: Optional[torch.Tensor] = None,
bias: Optional[torch.Tensor] = None, bias: Optional[torch.Tensor] = None,
cutlass_fp8_supported: bool = True, cutlass_fp8_supported: bool = True,
...@@ -192,7 +192,7 @@ def apply_int8_linear( ...@@ -192,7 +192,7 @@ def apply_int8_linear(
input: torch.Tensor, input: torch.Tensor,
weight: torch.Tensor, weight: torch.Tensor,
weight_scale: torch.Tensor, weight_scale: torch.Tensor,
input_scale: torch.Tensor, input_scale: Optional[torch.Tensor] = None,
bias: Optional[torch.Tensor] = None, bias: Optional[torch.Tensor] = None,
): ):
# ops.scaled_int8_quant supports both dynamic and static quant. # ops.scaled_int8_quant supports both dynamic and static quant.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment