支持kvacache fp8_e4m3/fp8_e5m2

支持kvacache fp8_e4m3/fp8_e5m2的RMS_ROPE_CONCAT

支持kvacache fp8_e4m3/fp8_e5m2
支持kvacache fp8_e4m3/fp8_e5m2的RMS_ROPE_CONCAT
442abc67 · xiabo · 0e5a20b3 · 442abc67
Commit 442abc67 authored Mar 24, 2026 by xiabo
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

vllm/model_executor/layers/attention/mla_attention.py vllm/model_executor/layers/attention/mla_attention.py +1 -1

No files found.
--- a/vllm/model_executor/layers/attention/mla_attention.py
+++ b/vllm/model_executor/layers/attention/mla_attention.py
@@ -2190,7 +2190,7 @@ class MLACommonImpl(MLACommonBaseImpl[M], Generic[M]):
        prefill_k_pe = k_pe[num_decode_tokens:]

        kv_cache_dtype_str: str | None = None
-        # if use_fused_rms_rope_concat:
+
        if not envs.VLLM_USE_LIGHTOP_RMS_ROPE_CONCAT:
            decode_q = q[:num_decode_tokens]
            prefill_q = q[num_decode_tokens:]