vllm/engine/arg_utils.py · a41351f363f3e7a212582e51b2c1c35c18aaa9df · OpenDAS / vllm_cscc

[Quantization][FP8] Add support for FP8 models with input_scale for output... · a41351f3

rasmith authored Apr 25, 2025


[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

a41351f3

arg_utils.py 77.6 KB

Replace arg_utils.py