GitLab

add VLLM_USE_FUSED_CACHE_QUANT_BMM_MLA to use fused rmsnorm + contiguous + rope(for dpsk-v3) + concat_and_cache_mla + q quant, control bmm(todo) + cat +mla (fp8)

update VLLM_USE_LIGHTOP_RMS_ROPE_CONCAT impl

set VLLM_USE_LIGHTOP_RMS_ROPE_CONCAT=1

set USE_FUSED_RMS_QUANT=1, USE_FUSED_SILU_MUL_QUANT=1 and  VLLM_SCHED_ENABLE_MINIMAL_INJECTION=1 for dpsk-v3

set VLLM_REJECT_SAMPLE_OPT=1 for dpsk-v3

support prefix cache on kme
fix the error in test_moe caused by moe align not supporting 511 and 211
multi-modal switching to torch implementation on z100l&k100

add VLLM_USE_LIGHTOP_MOE_SUM_MUL_ADD

set USE_FUSED_RMS_QUANT=1 and USE_FUSED_SILU_MUL_QUANT=1

add shared_output and routed_scaling_factor of CompressedTensorsW8A8Int8MoEMethod

[FIX] 修复mtp和VLLM_USE_TRITON_CAT不能一起开的bug

Signed-off-by: wang.yuqi <noooop@126.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: simon-mo <simon.mo@hey.com>

Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

support telechat2 and glm4 nn layout
remove log of request_id