vllm/model_executor/layers/quantization/fp8.py · a41351f363f3e7a212582e51b2c1c35c18aaa9df · OpenDAS / vllm_cscc

[Quantization][FP8] Add support for FP8 models with input_scale for output... · a41351f3

rasmith authored Apr 25, 2025


[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

a41351f3

fp8.py 37.9 KB

Replace fp8.py