Commits · 2ed8b6b3d0ccc5f577dc0d86845b63abc025468b · OpenDAS / vllm_cscc

16 Oct, 2025 25 commits
- Adding Warmup to Benchmark Serving (#26943) · 013abde6
  kimbochen authored Oct 16, 2025
```
Signed-off-by: Kimbo Chen <chentenghung@gmail.com>
```
  013abde6
- [Compressed Tensors] Always clone output for compile robustness (#26849) · a5464dcf
  Kyle Sayers authored Oct 16, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  a5464dcf
- Support block size of 256 used by Intel HPU (#26883) · ac3ed5a8
  Mandy Li authored Oct 16, 2025
```
Signed-off-by: mandy-li <mandy.j.li@intel.com>
```
  ac3ed5a8
- [gpt-oss][1/N] EZ: refactor serving_responses for modularity (#26948) · e6ba2000
  Andrew Xia authored Oct 16, 2025
```
Signed-off-by: Andrew Xia <axia@meta.com>
```
  e6ba2000
- Support `set` in the CLI generation (#27031) · aa255ff5
  Harry Mellor authored Oct 16, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aa255ff5
- Fix Qwen2.5 VL image grid docstring (#27033) · 7bb736d0
  ZiTian Zhao authored Oct 17, 2025
```
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
```
  7bb736d0
- [Model] Fix Qwen3VL mm mapping (#27027) · 9f4e3090
  Jee Jee Li authored Oct 17, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  9f4e3090
- [Feature] Add process_weights_after_loading to AttentionImpl (#26870) · 5afd3276
  rongfu.leng authored Oct 16, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  5afd3276
- [NIXL] Improve request_finished() debug logs (#25665) · 4a510ab4
  Mark McLoughlin authored Oct 16, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  4a510ab4
- [Benchmark] Show E2EL by default for pooling models (#27014) · 334535b6
  Cyrus Leung authored Oct 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  334535b6
- [Bugfix] Correct LayerNorm epsilon parameter in modernbert.py (#27008) · dcbb3f18
  bogdanm authored Oct 16, 2025
```
Signed-off-by: bogdanm <152898065+bogdan01m@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  dcbb3f18
- [MISC] fix import violations for re and triton modules (#26654) · 00417f4e
  Sungjae Lee authored Oct 16, 2025
```
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
```
  00417f4e
- [Model][Bugfix] fix ernie45 vl run failed from shared experts optimization (#26885) · e5192879
  CSWYF3634076 authored Oct 16, 2025
```
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
```
  e5192879
- [Chore] Separate out `vllm.utils.collections` (#26990) · d2740faf
  Cyrus Leung authored Oct 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2740faf
- [Benchmark] Use truncation by default for pooling benchmarks (#26992) · 17838e50
  Cyrus Leung authored Oct 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  17838e50
- [Hardware][CPU][PowerPC]Disable torch.compile() in toptopk sampling (#26987) · f7d318de
  Akash kaothalkar authored Oct 16, 2025
```
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
```
  f7d318de
- Deepseek-v3 Batch Invariant on 8xH100 (#26609) · 7d8975de
  Bram Wasti authored Oct 15, 2025
```
Signed-off-by: Bram Wasti <bwasti@meta.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  7d8975de
- [PERF] Qwen3-next MTP speedup (change bool mask indexing to index_select /... · 785d8b64
  Vadim Gimpelson authored Oct 16, 2025
```
[PERF] Qwen3-next MTP speedup (change bool mask indexing to index_select / index_copy to reduce d2h) (#26437)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
```
  785d8b64
- [Chore] Rename `utils` submodules (#26920) · f6cdc9a0
  Cyrus Leung authored Oct 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f6cdc9a0
- [BugFix] Work around graph partition x torch.compile cache issue (#26956) · 9b6504c3
  Richard Zou authored Oct 15, 2025
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
  9b6504c3
- [bugfix] Fix SP + PP without specifying compile size (#26955) · e19b16dd
  Angela Yi authored Oct 15, 2025
```
Signed-off-by: angelayi <yiangela7@gmail.com>
```
  e19b16dd
- [BUG] Allow runai_streamer_sharded in config check (#26958) · 582f2c6b
  ahao-anyscale authored Oct 15, 2025
```
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
```
  582f2c6b
- [ROCm][FEAT] Fuse DeepSeek shared experts into AITER fused_moe ops (#24097) · 13170343
  kliuae authored Oct 16, 2025
```
Signed-off-by: chenjun <junchen2@amd.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: valarLip <103567126+valarLip@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
```
  13170343
- [Bugfix] reasoning_parser parameter handling in run_batch.py (#26225) · 0ecc553e
  InChang Jeong authored Oct 16, 2025
```
Signed-off-by: inc-jeong <inc.jeong@navercorp.com>
Signed-off-by: InChang Jeong <inc.jeong@navercorp.com>
Co-authored-by: USER <user@AL02367916.local>
```
  0ecc553e
- [Qwen3-Next] Add tuned MoE config for Qwen3-Next FP8 on H100 tp2 (#26887) · f96bc364
  felixzhu555 authored Oct 15, 2025
```
Signed-off-by: Felix Zhu <felixzhu555@gmail.com>
```
  f96bc364
15 Oct, 2025 15 commits

Move query quantization to attention layer for Flashinfer & Triton. (#26534) · 0a9ef0cf

Adrian Abeyta authored Oct 15, 2025


Signed-off-by: adabeyta <aabeyta@redhat.com>
Signed-off-by: Adrian Abeyta <aabeyta@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

0a9ef0cf

[Bug] Temporally Disable `VLLM_ALLREDUCE_USE_SYMM_MEM` by Default (#26925) · e5b438a2
Wentao Ye authored Oct 15, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
e5b438a2

support flashinfer_fp4 moe for 5090 gpu (#26669) · 0b99f5d3

XiaobingZhang authored Oct 16, 2025


Signed-off-by: XiaobingSuper <xiaobingzhangupc@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

0b99f5d3

[NVIDIA] Add support for cudnn fp4 gemm via flashinfer (#26107) · de92d916

Kaixi Hou authored Oct 15, 2025


Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

de92d916

[ModelOpt] Remove NVFP4 MoE K%16==0 constraint (#26891) · d7963752
XiaobingZhang authored Oct 16, 2025
```
Signed-off-by: XiaobingSuper <xiaobingzhangupc@gmail.com>
```
d7963752

[Feature]: Use pydantic validation in observability.py config (#26637) · 14f84563

Sam/Samuel authored Oct 16, 2025

Signed-off-by: Samuel Wu <cernunnos1710@gmail.com>
Signed-off-by: Sam/Samuel <57896620+cern1710@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

14f84563

Olmo 3 tool parser and tests (#26143) · 4794c2bd
Pradeep Dasigi authored Oct 15, 2025
```
Signed-off-by: Pradeep Dasigi <pradeepd@allenai.org>
```
4794c2bd
Lower sevarity of log when model info cache misses due to exception (#26917) · d3cbaa08
Harry Mellor authored Oct 15, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
d3cbaa08
[Chore] Separate out `vllm.utils.async_utils` (#26913) · 828523ad
Cyrus Leung authored Oct 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
828523ad
[Chore] Separate out `vllm.utils.func` (#26904) · 136a17fe
Cyrus Leung authored Oct 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
136a17fe

[BugFix] Patch inductor memory plan logic (#26878) · f5743833

Boyuan Feng authored Oct 15, 2025

Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

f5743833

[Misc] rename torch_dtype to dtype (#26695) · 8f4b313c
wangxiyuan authored Oct 15, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
8f4b313c
[Misc] Remove `isort` and `yapf` ignores (#26888) · f93e3480
Cyrus Leung authored Oct 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
f93e3480
[Model][2/N] Improve all pooling task | Support multi-vector retrieval (#25370) · f54f8512
wang.yuqi authored Oct 15, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
f54f8512

[Lora]Load tuned multi-lora kernel configs from json files (#26319) · d4d1a602

li2haipeng authored Oct 15, 2025


Signed-off-by: li2haipeng <44383182+li2haipeng@users.noreply.github.com>
Signed-off-by: Haipeng Li <li2haipeng@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

d4d1a602