Commits · eeb135eb87277b388e1a2c97ff9c13fcd94fc6ae · OpenDAS / vllm_cscc

17 Sep, 2025 1 commit
- [Core] Use `CpuGpuBuffer` for block table tensors (#24795) · eeb135eb
  Nick Hill authored Sep 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  eeb135eb
16 Sep, 2025 7 commits

fp8 kv cache support fix for torch.compile (#22758) · 3053a22b

Aleksandr Malyshev authored Sep 16, 2025

Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>

3053a22b

[Bugfix] Fix unable to run encoder model when disable_hybrid_kv_cache_manager is true (#24571) · faa7a5da
lianyibo authored Sep 17, 2025
```
Signed-off-by: lianyibo <lianyibo1@kunlunit.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
```
faa7a5da

[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) · 56793995

Sage Moore authored Sep 16, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

56793995

[Bugfix] Fix sequence parallelism bug when enable pipeline parallelism (#24021) · 17871983
cascade authored Sep 15, 2025
```
Signed-off-by: cascade812 <cascade812@outlook.com>
```
17871983
Remove V0 Encoder-Decoder Support (#24907) · 759ef49b
Woosuk Kwon authored Sep 15, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
759ef49b
[Metrics] Hide deprecated metrics with gpu_ prefix (#24245) · 2942970d
Mark McLoughlin authored Sep 16, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
2942970d
[Bug] Fix `is_flashmla_supported` Check Error (#24774) · b42566f4
Wentao Ye authored Sep 15, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
b42566f4

15 Sep, 2025 6 commits
- [Performance] Remove redundant clone() calls in cutlass_mla (#24891) · aae725af
  Alexander Matveev authored Sep 15, 2025
  
  aae725af
- [gpt-oss] Add IncompleteDetails to ResponsesRepsonse (#24561) · 25aba2b6
  Andrew Xia authored Sep 15, 2025
```
Signed-off-by: Andrew Xia <axia@meta.com>
```
  25aba2b6
- [XPU] Set consistent default KV cache layout (#24745) · 2e41f5ab
  Nicolò Lucchesi authored Sep 15, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  2e41f5ab
- [P/D]`kv_output_aggregator` support P TP > D TP (#23917) · 8de261b0
  Chao Lei authored Sep 15, 2025
```
Signed-off-by: LCAIZJ <leichao139636@163.com>
Co-authored-by: leichao.lc <leichao.lc@antgroup.com>
```
  8de261b0
- [Misc] rename interval to max_recent_requests (#24229) · 59e17dd4
  Ning Xie authored Sep 15, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  59e17dd4
- [kv cache] update num_free_blocks in the end (#24228) · 3f331398
  Ning Xie authored Sep 15, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  3f331398
14 Sep, 2025 3 commits
- [Hybrid Allocator] Support Pipeline Parallel (#23974) · 8e5cdcda
  Chen Zhang authored Sep 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  8e5cdcda
- [Spec Decoding]Support Spec Decoding Metrics in DP Mode (#24049) · 90f3f7d7
  wuhang authored Sep 15, 2025
```
Signed-off-by: wuhang <wuhang6@huawei.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
  90f3f7d7
- [Chore] Minor simplification for non-PP path (#24810) · 3e903b6c
  Woosuk Kwon authored Sep 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  3e903b6c
13 Sep, 2025 2 commits
- [Misc] Correct an outdated comment. (#24765) · 4dad72f0
  Russell Bryant authored Sep 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4dad72f0
- [Bugfix] Fix GPUModelRunner has no attribute lora_manager (#24762) · 99bfef84
  Jee Jee Li authored Sep 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  99bfef84
12 Sep, 2025 10 commits

[Core] Support async scheduling with uniproc executor (#24219) · 4fdd6f5c

Nick Hill authored Sep 12, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

4fdd6f5c

[Qwen3Next] Fixes the cuda graph capture conditions under large batch sizes (#24660) (#24667) · 8226dd56
Tao He authored Sep 13, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
```
8226dd56
[Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705) · 7ba32aa6
Matthew Bonanni authored Sep 12, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
```
7ba32aa6
[Bugfix] Fix incompatibility between #20452 and #24548 (#24754) · 010acc6e
Cyrus Leung authored Sep 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
010acc6e
[UX] Remove AsyncLLM torch profiler disabled log (#24609) · 9d2a4460
Michael Goin authored Sep 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
9d2a4460
[Core] Shared memory based object store for Multimodal data caching and IPC (#20452) · a5b84f1c
dongluw authored Sep 12, 2025
```
Signed-off-by: donglu <donglu@cohere.com>
```
a5b84f1c
[Doc]: fix typos in various files (#24726) · bcb06d7b
Didier Durand authored Sep 12, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
bcb06d7b
[Multimodal] Remove legacy multimodal fields in favor of MultiModalFeatureSpec (#24548) · 0377802c
Flora Feng authored Sep 12, 2025
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
0377802c
[Kernel] [CPU] refactor `cpu_attn.py:_run_sdpa_forward` for better memory access (#24701) · 7a1c4025
Ignacio Sica authored Sep 12, 2025
```
Signed-off-by: ignaciosica <mignacio.sica@gmail.com>
```
7a1c4025

[V1] feat:add engine v1 tracing (#20372) · 40b6c912

RichardoMu authored Sep 12, 2025


Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>
Signed-off-by: Ye Zhang <zhysishu@gmail.com>
Signed-off-by: RichardoMu <44485717+RichardoMrMu@users.noreply.github.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Mu Huai <tianbowen.tbw@antgroup.com>
Co-authored-by: Ye Zhang <zhysishu@gmail.com>
Co-authored-by: Benjamin Bartels <benjamin@bartels.dev>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: 瑜琮 <ly186375@antfin.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>

40b6c912

11 Sep, 2025 9 commits
- [CI/Build] Add bc-linter to vLLM CI (#21234) · 7d465199
  Zhewen Li authored Sep 11, 2025
```
Signed-off-by: zhewenli <zhewenli@meta.com>
```
  7d465199
- [flashinfer] [kernel] support for fp8 kv cache for trtllm prefill attention (#24197) · e42af78b
  Xiaozhu Meng authored Sep 11, 2025
```
Signed-off-by: Xiaozhu <mxz297@gmail.com>
```
  e42af78b
- Update Spec Decode metrics to include drafted and accepted token throughput (#24127) · 79ac59f3
  Andrew Xia authored Sep 11, 2025
```
Signed-off-by: Andrew Xia <axia@meta.com>
```
  79ac59f3
- fix some typos (#24616) · e26fef83
  co63oc authored Sep 12, 2025
```
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
```
  e26fef83
- Allow users to specify kv cache memory size (#21489) · 94e6b2d5
  Boyuan Feng authored Sep 11, 2025
```
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  94e6b2d5
- Move `LoRAConfig` from `config/__init__.py` to `config/lora.py` (#24644) · 5f5271f1
  Harry Mellor authored Sep 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  5f5271f1
- Add the support for the qwen3 next model (a hybrid attention model). (#24526) · e93f4cc9
  Tao He authored Sep 11, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  e93f4cc9
- [Doc]: fixing doc typos (#24635) · e2b1f863
  Didier Durand authored Sep 11, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  e2b1f863
- [BugFix] Fix pipeline parallel (#24621) · e2d8c27f
  Nick Hill authored Sep 10, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  e2d8c27f
10 Sep, 2025 2 commits

[Kernel] Flashinfer MLA (trtllm-gen) decode kernel integration (#21078) · dcb28a33

Hanjie Qiu authored Sep 10, 2025


Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

dcb28a33

[Perf] Warmup FlashInfer attention during startup (#23439) · fba78565

Michael Goin authored Sep 10, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>

fba78565