Commits · 896e41ae04d18b0f984eefbb41b920aa7505f5d1 · OpenDAS / vllm_cscc

17 Nov, 2025 8 commits

[Misc] Fix wrong comment in scheduler (#28880) · 552cac95
Zhuohan Li authored Nov 17, 2025
```
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
```
552cac95
[BugFix] Corner case that could cause out-of-sync with external launcher mode and dp >1 (#28774) · 61485844
Bangsheng Tang authored Nov 17, 2025

61485844
[BugFix] Fix PP performance and PP kv connector output regression (#28768) · 7765e5ba
Nick Hill authored Nov 17, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
7765e5ba

[Core] Async Scheduling X Spec Decoding Compatibility (#24799) · d8874c61

Ronald authored Nov 18, 2025


Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>

d8874c61

[BugFix] Temporary fix for IMA with MTP = 2 and full-cg (#28315) · 64e39d66
Lucas Wilkinson authored Nov 17, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
64e39d66

[Metrics] Fix KV cache usage percent metric multiproc (#28792) · d4acf518

Jae-Won Chung authored Nov 17, 2025



The `vllm:kv_cache_usage_perc` Gauge metric is missing `multiprocess_mode="mostrecent"` and ends up returning

```
vllm:kv_cache_usage_perc{engine="0",model_name="Qwen/Qwen3-VL-8B-Instruct",pid="277"} 0.0
vllm:kv_cache_usage_perc{engine="0",model_name="Qwen/Qwen3-VL-8B-Instruct",pid="275"} 0.0
vllm:kv_cache_usage_perc{engine="0",model_name="Qwen/Qwen3-VL-8B-Instruct",pid="273"} 0.6530455880475035
...
```

The deprecated `vllm:gpu_cache_usage_perc` Gauge metric has `multiprocess_mode="mostrecent"`.
Signed-off-by: Jae-Won Chung <jwnchung@umich.edu>

d4acf518

[CPU][Bugfix] Fix _to_list in CPU model runner (#28824) · 577bb34f
Li, Jiang authored Nov 17, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
577bb34f
[ROCm][Qwen3-32B] Fix AITER MHA accuracy issue cause by #25763 (#28670) · 60e089f0
Xiake Sun authored Nov 17, 2025
```
Signed-off-by: Xiake Sun <xiake.sun@amd.com>
```
60e089f0

16 Nov, 2025 3 commits

[BugFix] Fix async scheduling + chunked prefill + preemption (#28787) · 80b6080d
Nick Hill authored Nov 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
80b6080d

[V1] Support MP Executor for multi node distributed inference (#23691) · b316ac65

Lucia Fang authored Nov 16, 2025


Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

b316ac65

[Model] Allow users to control skip reading cache per request. (#28194) · a55b6463
wang.yuqi authored Nov 16, 2025
```
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
```
a55b6463

15 Nov, 2025 9 commits
- [BugFix] Fix `AssertionError: DCP not support reorder_batch_threshold > 1 now.` (#28751) · be263f76
  Lucas Wilkinson authored Nov 15, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  be263f76
- [Doc]: fix typos in various files (#28567) · 2bb4435c
  Didier Durand authored Nov 15, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  2bb4435c
- Add support for Eagle with separate lm-head and embed_tokens layers (#28549) · e439c784
  Eldar Kurtić authored Nov 15, 2025
```
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
```
  e439c784
- Allow Gemma3 to take image embeddings (#28483) · cb15ee28
  tingtinggithub authored Nov 15, 2025
```
Signed-off-by: tingtinggithub <streamttt@gmail.com>
```
  cb15ee28
- [Misc] Make `SchedulerConfig.max_model_len` init-only (#28733) · 638e4196
  Cyrus Leung authored Nov 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  638e4196
- [RL] [V1] Remove unused device argument from reset_kv_cache (#28766) · dd6ac1c2
  Zhuohan Li authored Nov 14, 2025
```
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
```
  dd6ac1c2
- [Redo] #26368 (#28771) · 98b4d389
  Cyrus Leung authored Nov 15, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  98b4d389
- Revert "[Core] Performance: Use list[np.ndarray] instead of list[list… (#28773) · ac86bff8
  Nick Hill authored Nov 14, 2025
  
  ac86bff8
- [Core] Performance: Use list[np.ndarray] instead of list[list[int]] for output... · 186352b2
  Jialin Ouyang authored Nov 14, 2025
```
[Core] Performance: Use list[np.ndarray] instead of list[list[int]] for output tokens for GC optimization (#26368)
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  186352b2
14 Nov, 2025 12 commits
- [Log] Save profiler results to file instead of stdout (#28144) · ba041d98
  rasmith authored Nov 14, 2025
```
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>
```
  ba041d98
- Avoid bytecode hook and simplify TorchCompileWrapperWithCustomDipatch (#25110) · 2e0ad629
  Laith Sakka authored Nov 14, 2025
```
Signed-off-by: Laith Sakka <lsakka@meta.com>
```
  2e0ad629
- [Chore] Rename `SchedulerConfig.chunked_prefill_enabled` (#28735) · e2741f6c
  Cyrus Leung authored Nov 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e2741f6c
- [DisaggEverything] Tokens in<>out `/generate` endpoint (#24261) · 6f1e7f72
  Nicolò Lucchesi authored Nov 14, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6f1e7f72
- [Bugfix][Nixl] Fix kernel physical<>logical block_size issue (#28677) · 96b23b8e
  Nicolò Lucchesi authored Nov 14, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  96b23b8e
- [BugFix] Fix FA3 IMA with FULL_AND_PIECEWISE and cascade attention (default) (#28702) · db56a599
  Lucas Wilkinson authored Nov 14, 2025
  
  db56a599
- Fix KV sharing fast prefill with cudagraph enabled (#28537) · 9324e102
  Yong Hoon Shin authored Nov 14, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  9324e102
- [DCP] Support Decode Context Parallel (DCP) for GQA with Flashinfer (#25438) · 4516d44b
  Jingchun Gao authored Nov 14, 2025
```
Signed-off-by: gaojc <1055866782@qq.com>
Signed-off-by: Jingchun Gao <gaojingchun1@huawei.com>
Signed-off-by: Jingchun Gao <63247409+gjc0824@users.noreply.github.com>
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: gaojingchun (A) <g00955623@china.huawei.com>
Co-authored-by: Jingchun Gao <gaojingchun1@huawei.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
```
  4516d44b
- [Metrics] Log number of preempted requests (#28522) · ecf8230d
  lyn610 authored Nov 14, 2025
```
Add tracking and periodic logging for the number of preempted requests in the
metrics logger. This helps monitor system behavior under load.
Signed-off-by: Yining Liu <610lyn@gmail.com>
```
  ecf8230d
- [BugFix] Fix multi-modal async scheduling race condition (#28706) · bc3e4306
  Nick Hill authored Nov 14, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  bc3e4306
- use default CCL_ZE_IPC_EXCHANGE (#28700) · 529cea34
  Yan Ma authored Nov 14, 2025
```
Signed-off-by: Yan Ma <yan.ma@intel.com>
```
  529cea34
- [Misc] Remove `warn_for_unimplemented_methods` (#28613) · 01bea115
  Cyrus Leung authored Nov 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  01bea115
13 Nov, 2025 8 commits

[CI] Bug: Fix ci entrypoint pooling (#28684) · e64011f2
Wentao Ye authored Nov 13, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
e64011f2

[bugfix] correct local_chunk_len for DCP in reorg_kvcache with long context (#28526) · 968060c1

Qiu authored Nov 14, 2025


Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

968060c1

[Perf] Support stream interval for reducing host overhead (#27869) · 5d6ce2b9

elvischenv authored Nov 14, 2025


Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

5d6ce2b9

[Attention][Bugfix] Fix FA sink support (#28660) · f9f3b596
Matthew Bonanni authored Nov 13, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
f9f3b596

[Bugfix] Fix validate model input for decoder models (#27099) · 119c4927

Yannick Schnider authored Nov 13, 2025


Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

119c4927

[CI Failure] Fix backend selection for encoder-only models (#28534) · 07a606aa
Huamin Li authored Nov 13, 2025
```
Signed-off-by: Huamin Li <3ericli@gmail.com>
```
07a606aa
[ROCm][BugFix]Fix `get_cu_count` in rocm_aiter_fa.py (#28618) · 8da2f28f
Pleaplusone authored Nov 13, 2025
```
Signed-off-by: ganyi <ygan@amd.com>
```
8da2f28f
[Bugfix] Prevent crash on empty grammar string (#28210) · 4504e802
tjandy98 authored Nov 13, 2025
```
Signed-off-by: tjandy98 <3953059+tjandy98@users.noreply.github.com>
```
4504e802