Commits · 5a16fa614c78e1f401125cd7384c602f83cb2160 · OpenDAS / vllm_cscc

09 Aug, 2025 8 commits

[Model] Gemma3n MM (#20495) · 5a16fa61

Nicolò Lucchesi authored Aug 09, 2025


Signed-off-by: ShriKode <shrikode@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Roger Wang <hey@rogerw.me>
Co-authored-by: ShriKode <shrikode@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.me>

5a16fa61

GLM-4.5V with new class name at transformers (#22520) · a6022e6f

Yuxuan Zhang authored Aug 09, 2025


Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

a6022e6f

[TPU] Add support for online w8a8 quantization (#22425) · 9a0c5ded
Kyuyeun Kim authored Aug 08, 2025
```
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
```
9a0c5ded
Fix loading of quantized BigCode models (#22463) · 10a02535
Eldar Kurtić authored Aug 09, 2025
```
Signed-off-by: Eldar Kurtic <eldar@neuralmagic.com>
```
10a02535
[Misc] Use config definitions from Transformers library (#21913) · 65552b47
Cyrus Leung authored Aug 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
65552b47
[V1] [Hybrid] Support Minimax-Text-01 in V1 (#22151) · 6ade99ea
Thomas Parnell authored Aug 09, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
6ade99ea

Implicit language-model-only mode via limit-mm-per-prompt (#22299) · 08b751ba

Roger Wang authored Aug 08, 2025


Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Signed-off-by: Junhao Li <junhao@ubicloud.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Signed-off-by: Linkun <github@lkchen.net>
Co-authored-by: Ning Xie <andy.xning@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
Co-authored-by: Zhiyu <zhiyuc@nvidia.com>
Co-authored-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
Co-authored-by: Junhao Li <streaver91@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Yuxuan Zhang <2448370773@qq.com>
Co-authored-by: ZiTian Zhao <zitian.zhao@tencentmusic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Hong Hanh <hanh.usth@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: lkchen <github@lkchen.net>

08b751ba

[Bugfix] Fix ModernBert cuda graph capturing in v1 (#21901) · 429e4e2d
Isotr0py authored Aug 09, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
```
429e4e2d

08 Aug, 2025 11 commits
- [Misc] DeepGEMM : Avoid JIT generation in the hot-path (#22215) · f703b923
  Varun Sundar Rabindranath authored Aug 08, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  f703b923
- [gpt-oss] guard import when triton kernel is not installed (#22529) · f756a682
  Yongye Zhu authored Aug 08, 2025
```
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f756a682
- [gpt-oss] triton kernel mxfp4 (#22421) · e789cad6
  Yongye Zhu authored Aug 08, 2025
```
Signed-off-by: <zyy1102000@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
```
  e789cad6
- [bugfix] Fix Llama3/4 issues caused by FlashInfer 0.2.10 (#22426) · af473f0a
  Po-Han Huang (NVIDIA) authored Aug 08, 2025
```
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
```
  af473f0a
- Fix pre-commit (#22487) · 157f9c13
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  157f9c13
- Optimize MiniCPMO mask creation with vectorized implementation (#22464) · 6f287915
  ZiTian Zhao authored Aug 08, 2025
```
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
```
  6f287915
- not tie_word_embeddings for glm-4.5 and glm-4.5v (#22460) · c152e2a8
  Yuxuan Zhang authored Aug 08, 2025
```
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
```
  c152e2a8
- [Bugfix] Fix RuntimeError: Index put requires the source and destination dtypes match (#22065) · 17eaaef5
  Chauncey authored Aug 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  17eaaef5
- Fix Flashinfer CUTLASS MOE Allgather (#21963) · b2c8ce57
  Shu Wang authored Aug 07, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  b2c8ce57
- Support Tensorrt-LLM MoE fp4 for low-latency (#21331) · a3b9c17b
  Shu Wang authored Aug 07, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
```
  a3b9c17b
- Add ModelOpt Qwen3 nvfp4 support (#20101) · d57dc236
  Zhiyu authored Aug 07, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
  d57dc236
07 Aug, 2025 16 commits

[Core] Simplify mm processing cache (#22457) · 8c9da6be
Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
8c9da6be
[Bugfix] Add missing `packed_modules_mapping` to `DeepseekV2ForCausalLM` (#22352) · 7e0b1218
fxmarty-amd authored Aug 07, 2025
```
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
```
7e0b1218
[Misc] Enhance code formatting in mxfp4.py (#22423) · 136825de
Woosuk Kwon authored Aug 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
136825de

Add H20-3e fused MoE kernel tuning configs for GLM-4.5 (#22433) · c2dba2db

JaceyShao authored Aug 07, 2025


Signed-off-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
Co-authored-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>

c2dba2db

[Misc] Support routing logic simulation (#21990) · 82216dc2

Ming Yang authored Aug 06, 2025

Signed-off-by: Ming Yang <minos.future@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

82216dc2

[Model] Switch to Fused RMS norm in Qwen2.5_VL model. (#22184) · cbc8457b

vllmellm authored Aug 07, 2025


Signed-off-by: kf <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaavllm <tunjian.tan@amd.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: kf <kuanfu.liu@embeddedllm.com>

cbc8457b

[Bugfix] EPLB load statistics problem (#22167) · 4be02a37

WeiQing Chen authored Aug 07, 2025


Signed-off-by: ycyaw66 <497410282@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: ycyaw66 <497410282@qq.com>

4be02a37

[Bugfix] Add proper comparison for package versions (#22314) · 609b533c
Syed Muhammad Bin Asif authored Aug 07, 2025
```
Signed-off-by: Syed Muhammad Bin Asif <syedmba7@connect.hku.hk>
```
609b533c
[Bugfix] Fix wrong method name in Intern-S1 image processor (#22417) · 04cf435d
Cyrus Leung authored Aug 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
04cf435d
[Qwen3] Enable dual-chunk-attention support for Qwen3 models. (#21924) · 7377131a
Tao He authored Aug 07, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
```
7377131a

[Attention] Support multiple attention metadata builders per kv_cache_spec +... · 1dc8a70b

Lucas Wilkinson authored Aug 06, 2025


[Attention] Support multiple attention metadata builders per kv_cache_spec  + proper local attention no hybrid kv cache fix (#21588)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

1dc8a70b

[model] Support MiniCPM-V 4.0 (#22166) · 41b67f42
tc-mb authored Aug 07, 2025
```
Co-authored-by: imning3 <hbning@pku.edu.cn>
```
41b67f42

Fix trtllm-gen attention env and add attention sink (#22378) · 9a3835aa

Lain authored Aug 06, 2025


Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: Lain <fusiyuan2000@hotmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

9a3835aa

[gpt-oss] fix model config with hf_config (#22401) · 5c7cc33f
Yongye Zhu authored Aug 06, 2025
```
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
```
5c7cc33f
[Bug] Fix B200 DeepGEMM E8M0 Accuracy Issue (#22399) · eec890c1
Wentao Ye authored Aug 06, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
eec890c1
[v1] - Mamba1 Attention Metadata (#21249) · 46a13949
Asaf Joseph Gardin authored Aug 07, 2025
```
Signed-off-by: asafg <asafg@ai21.com>
Co-authored-by: asafg <asafg@ai21.com>
```
46a13949

06 Aug, 2025 5 commits

[gpt-oss] flashinfer mxfp4 (#22339) · 31f09c61

Yongye Zhu authored Aug 06, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

31f09c61

[GptOss] Add GptOss reasoning parser to support structure output (#22322) · a47e6ffe

Chen Zhang authored Aug 05, 2025


Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

a47e6ffe

Add GPT-OSS model code and config [1/N] (#22327) · de98252f
Woosuk Kwon authored Aug 05, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
de98252f

Update transformers to `v4.55` (#21931) · 796bae07

Harry Mellor authored Aug 06, 2025


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

796bae07

[Bugfix] Fix MoE BNB version (#22260) · 8e6c7e87
Jee Jee Li authored Aug 06, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
8e6c7e87