Commits · 41d3071918bd6dcb6259bac8a617cfed45d42707 · OpenDAS / vllm_cscc

16 Oct, 2025 17 commits

[NVIDIA] [Perf] Update to leverage flashinfer trtllm FP4 MOE throughput kernel (#26714) · 41d30719
jiahanc authored Oct 16, 2025
```
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
41d30719
Refactor Transformers backend to use mixins (#26906) · fb5e10d3
Harry Mellor authored Oct 16, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
fb5e10d3
[small][batch invariance] Rename the env and internal flags to simplify usage (#26855) · b2f78cba
Bram Wasti authored Oct 16, 2025
```
Signed-off-by: Bram Wasti <bwasti@meta.com>
```
b2f78cba

[Feature] Migrate DeepGEMM API from `get_m_alignment_for_contiguous_layout` to... · b3dda72c

Wentao Ye authored Oct 16, 2025


[Feature] Migrate DeepGEMM API from `get_m_alignment_for_contiguous_layout` to `get_mk_alignment_for_contiguous_layout` (#26935)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

b3dda72c

[GPTOSS][DP/EP][Marlin] Enable GPTOSS Batched DP/EP using Marlin kernels (#25997) · fb0571b0
Varun Sundar Rabindranath authored Oct 16, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
fb0571b0
[Compressed Tensors] Always clone output for compile robustness (#26849) · a5464dcf
Kyle Sayers authored Oct 16, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
a5464dcf
Fix Qwen2.5 VL image grid docstring (#27033) · 7bb736d0
ZiTian Zhao authored Oct 17, 2025
```
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
```
7bb736d0
[Model] Fix Qwen3VL mm mapping (#27027) · 9f4e3090
Jee Jee Li authored Oct 17, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
9f4e3090

[Bugfix] Correct LayerNorm epsilon parameter in modernbert.py (#27008) · dcbb3f18

bogdanm authored Oct 16, 2025


Signed-off-by: bogdanm <152898065+bogdan01m@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

dcbb3f18

[MISC] fix import violations for re and triton modules (#26654) · 00417f4e

Sungjae Lee authored Oct 16, 2025


Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>

00417f4e

[Model][Bugfix] fix ernie45 vl run failed from shared experts optimization (#26885) · e5192879
CSWYF3634076 authored Oct 16, 2025
```
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
```
e5192879
[Chore] Separate out `vllm.utils.collections` (#26990) · d2740faf
Cyrus Leung authored Oct 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
d2740faf

Deepseek-v3 Batch Invariant on 8xH100 (#26609) · 7d8975de

Bram Wasti authored Oct 15, 2025


Signed-off-by: Bram Wasti <bwasti@meta.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

7d8975de

[PERF] Qwen3-next MTP speedup (change bool mask indexing to index_select /... · 785d8b64

Vadim Gimpelson authored Oct 16, 2025


[PERF] Qwen3-next MTP speedup (change bool mask indexing to index_select / index_copy to reduce d2h) (#26437)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>

785d8b64

[Chore] Rename `utils` submodules (#26920) · f6cdc9a0
Cyrus Leung authored Oct 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
f6cdc9a0

[ROCm][FEAT] Fuse DeepSeek shared experts into AITER fused_moe ops (#24097) · 13170343

kliuae authored Oct 16, 2025


Signed-off-by: chenjun <junchen2@amd.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: valarLip <103567126+valarLip@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>

13170343

[Qwen3-Next] Add tuned MoE config for Qwen3-Next FP8 on H100 tp2 (#26887) · f96bc364
felixzhu555 authored Oct 15, 2025
```
Signed-off-by: Felix Zhu <felixzhu555@gmail.com>
```
f96bc364

15 Oct, 2025 14 commits
- support flashinfer_fp4 moe for 5090 gpu (#26669) · 0b99f5d3
  XiaobingZhang authored Oct 16, 2025
```
Signed-off-by: XiaobingSuper <xiaobingzhangupc@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  0b99f5d3
- [NVIDIA] Add support for cudnn fp4 gemm via flashinfer (#26107) · de92d916
  Kaixi Hou authored Oct 15, 2025
```
Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  de92d916
- [ModelOpt] Remove NVFP4 MoE K%16==0 constraint (#26891) · d7963752
  XiaobingZhang authored Oct 16, 2025
```
Signed-off-by: XiaobingSuper <xiaobingzhangupc@gmail.com>
```
  d7963752
- Lower sevarity of log when model info cache misses due to exception (#26917) · d3cbaa08
  Harry Mellor authored Oct 15, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d3cbaa08
- [Chore] Separate out `vllm.utils.func` (#26904) · 136a17fe
  Cyrus Leung authored Oct 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  136a17fe
- [Misc] rename torch_dtype to dtype (#26695) · 8f4b313c
  wangxiyuan authored Oct 15, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  8f4b313c
- [Misc] Remove `isort` and `yapf` ignores (#26888) · f93e3480
  Cyrus Leung authored Oct 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f93e3480
- [Model][2/N] Improve all pooling task | Support multi-vector retrieval (#25370) · f54f8512
  wang.yuqi authored Oct 15, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  f54f8512
- [Fix] Remove divisibility requirement between num_kv_heads and tp_size in bailing_moe (#26876) · 5c3bae1a
  ant-yy authored Oct 15, 2025
```
Signed-off-by: vito.yy <vito.yy@antgroup.com>
```
  5c3bae1a
- [Deepseek-V3.2][Kernel] Integrate cuda indexer k cache gather (#26456) · f5ed68ef
  Yongye Zhu authored Oct 15, 2025
```
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
```
  f5ed68ef
- [DSA][MLA] Tiny refactor on DeepSeek to make it reusable for different backends (#26656) · 302ef403
  Mengqing Cao authored Oct 15, 2025
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  302ef403
- [Bugfix][Multi Modal] Fix incorrect Molmo token processing (#26873) · 8865da15
  sangho.lee authored Oct 15, 2025
```
Signed-off-by: sanghol <sanghol@allenai.org>
```
  8865da15
- [Bugfix] Fix qwen3-omni audio truncation issue (#26815) · 8c851f6d
  Isotr0py authored Oct 15, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  8c851f6d
- [Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change ... · 96b9aa5a
  Morrison Turnansky authored Oct 14, 2025
```
[Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change  compilation level to compilation mode, deprecation compilation level (#26355)
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  96b9aa5a
14 Oct, 2025 9 commits
- [CI Failure] Fix torchao dep failure for Quantization Test (#26824) · 7e0ef408
  Michael Goin authored Oct 14, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  7e0ef408
- Added MoE configs for llama 4, H200 device with tp=4/8 tuning (#26837) · 0e658189
  Dhruvil Bhatt authored Oct 14, 2025
```
Signed-off-by: Dhruvil Bhatt <bhattdbh@amazon.com>
```
  0e658189
- llama4_vision_rope: add HIP override to accept (q, k) and avoid (positions, q, k) mismatch (#26790) · 87efc681
  Huamin Li authored Oct 14, 2025
```
Signed-off-by: Huamin Li <3ericli@gmail.com>
```
  87efc681
- [Kernel][MoE] Add MoE tunings for GLM 4.6-FP8 and GLM 4.5 Air on NVidia B200 (#26818) · aba48f7d
  Ze'ev Klapow authored Oct 14, 2025
  
  aba48f7d
- Adjusted the model order of the model registration file (#26798) · e9f1b8c9
  汪志鹏 authored Oct 14, 2025
```
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
```
  e9f1b8c9
- [Chore] Remove `SupportsV0Only` interface and update supported models docs (#26783) · 9c4cb683
  Cyrus Leung authored Oct 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9c4cb683
- [Model] Use merge_by_field_config for MM models (O-P) (#26776) · 74704d45
  Cyrus Leung authored Oct 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  74704d45
- [Bugfix] Standardize merging multimodal embeddings (#26771) · d2f816d6
  Cyrus Leung authored Oct 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f816d6
- [Model][Bugfix]fix ernie45 load failed due to ernie45 eplb code (#26684) · 01ad27fa
  CSWYF3634076 authored Oct 14, 2025
```
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
```
  01ad27fa