GitLab

set USE_FUSED_RMS_QUANT=1, USE_FUSED_SILU_MUL_QUANT=1 and  VLLM_SCHED_ENABLE_MINIMAL_INJECTION=1 for dpsk-v3

set VLLM_REJECT_SAMPLE_OPT=1 for dpsk-v3

add VLLM_USE_LIGHTOP_MOE_SUM_MUL_ADD

set USE_FUSED_RMS_QUANT=1 and USE_FUSED_SILU_MUL_QUANT=1

add shared_output and routed_scaling_factor of CompressedTensorsW8A8Int8MoEMethod

[FIX] 修复mtp和VLLM_USE_TRITON_CAT不能一起开的bug

Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: wang.yuqi <noooop@126.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: Andy Xie <andy.xning@gmail.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: simon-mo <simon.mo@hey.com>

Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>

Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>

[LoRA] Support dynamically initialize `packed_modules_mapping` for VLM with arbitrary components (#18987)
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Ashraf Mahgoub <ashymahg@amazon.com>
Co-authored-by: Rohith Nallamaddi <nalrohit@amazon.com>
Co-authored-by: FeliciaLuo <luof@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>

Signed-off-by: Chenyaaang <chenyangli@google.com>

Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>