Commits · 856589ed9aa7fbac86af8e86374e4f00ed601ecd · OpenDAS / vllm_cscc

31 Mar, 2026 3 commits
- [Refactor] Remove dead code in kv connector and model runner (#38383) · 856589ed
  Wentao Ye authored Mar 31, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  856589ed
- [torch.compile] Refactor Attention Quant Fusion Pass and Remove Boilerplate (#37373) · 077a9a8e
  BadrBasowid authored Apr 01, 2026
```
Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  077a9a8e
- [Bugfix][Async] Fix async spec decoding with hybrid models (#38556) · 757068dc
  Matthew Bonanni authored Mar 31, 2026
```
Signed-off-by: SandishKumarHN <sandishkumarhn@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: SandishKumarHN <sandishkumarhn@gmail.com>
```
  757068dc
30 Mar, 2026 5 commits

[Bugfix] Use null block (0) for padded block table entries (#35431) · bcc6f674

SandishKumarHN authored Mar 30, 2026


Signed-off-by: SandishKumarHN <sandish@fb.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>

bcc6f674

[Feat][Spec Decode] DFlash (#36847) · 494636b2
Benjamin Chislett authored Mar 30, 2026
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
494636b2
[Bugfix][CPU] Skip set_num_threads after thread binding (#38535) · 6557f493
Li, Jiang authored Mar 30, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
6557f493

Fix ambiguous num_blocks for hybrid attn mamba (#37236) · 1031c84c

Collin McCarthy authored Mar 30, 2026

Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>

1031c84c

[Mamba][Bugfix] Raise on insufficient cache blocks instead of silently capping... · cc06b4e8

Nicolò Lucchesi authored Mar 30, 2026


[Mamba][Bugfix] Raise on insufficient cache blocks instead of silently capping cudagraph sizes (#38270)
Signed-off-by: NickLucche <nlucches@redhat.com>

cc06b4e8

29 Mar, 2026 2 commits
- [QeRL] Fix online quantized reloading (#38442) · d28d86e8
  Kyle Sayers authored Mar 29, 2026
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  d28d86e8
- [Perf] Remove redundant device copies for CPU-only pooling token IDs, 48.9%... · 995dea13
  Wentao Ye authored Mar 29, 2026
```
[Perf] Remove redundant device copies for CPU-only pooling token IDs, 48.9% E2E throughput improvement (#38139)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  995dea13
27 Mar, 2026 1 commit
- [Model Runner V2] Rebuild attention metadata before eagle decode full… (#38311) · 384e4d5f
  Giancarlo Delfin authored Mar 27, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  384e4d5f
26 Mar, 2026 2 commits
- [Model Runner V2] Enable forcing a specific acceptance rate during rejection sampling (#38045) · c32e9760
  Giancarlo Delfin authored Mar 26, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  c32e9760
- Relocate Encoder CUDA graph manager (#38116) · 144030c8
  Woosuk Kwon authored Mar 25, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
```
  144030c8
25 Mar, 2026 1 commit

[Feature] EPLB Support for GPU Model Runner v2 (#37488) · d7e93e13

Wentao Ye authored Mar 25, 2026


Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>

d7e93e13

24 Mar, 2026 8 commits
- Make microbatch optimization (DBO) work with general models (#37926) · b73b5b06
  Junhao authored Mar 24, 2026
```
Signed-off-by: Junhao Li <junhao@ubicloud.com>
```
  b73b5b06
- [MRV2] Fix for DS v3.2 (#38030) · 4b53740d
  Woosuk Kwon authored Mar 24, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  4b53740d
- [Model Runner V2][Minor] Simplify PP logic (#38031) · 4e824d1c
  Nick Hill authored Mar 24, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  4e824d1c
- Fix Mamba state corruption from referencing stale block table entries (#37728) (#37728) (#37728) · c07e2ca6
  Ming Yang authored Mar 24, 2026
  
  c07e2ca6
- [Feature] limit thinking tokens (hard limit) (#20859) · 47318847
  Sungjae Lee authored Mar 25, 2026
```
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  47318847
- [Bugfix] Add replacement of _compute_slot_mapping_kernel on CPU (#37987) · 352b90c4
  Li, Jiang authored Mar 24, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  352b90c4
- [V0 Deprecation] Refactor kv cache from list to element (#37487) · c59a132f
  Wentao Ye authored Mar 23, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  c59a132f
- [Model Runner V2] Gather multimodal embeddings before draft model postprocess (#37932) · 8f4824b6
  Giancarlo Delfin authored Mar 23, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  8f4824b6
23 Mar, 2026 5 commits

[Async][Spec Decoding] Zero-bubble async scheduling + spec decoding (#32951) · fafe76b4

Matthew Bonanni authored Mar 23, 2026


Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: zhrrr <43847754+izhuhaoran@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>

fafe76b4

[MRV2] Consider spec decoding in warmup (#37812) · ffb5b32b

Woosuk Kwon authored Mar 23, 2026


Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>

ffb5b32b

[Bugfix] RoBERTa position_id accumulation in CUDA graph padding region (#37873) · 7151ae65
yanghui1-arch authored Mar 23, 2026
```
Signed-off-by: dass90 <3053034939@qq.com>
```
7151ae65
[Bugfix] Fix CPU backend crash in KV cache block zeroing (#37550) · 7938d121
DorBernsohn authored Mar 23, 2026
```
Signed-off-by: DorBernsohn <dor.bernsohn@gmail.com>
```
7938d121
[Feature] ViT Full CUDA Graph (#35963) · f85e479e
Baorun (Lauren) Mu authored Mar 23, 2026
```
Signed-off-by: Baorun Mu <bmu@nvidia.com>
```
f85e479e

22 Mar, 2026 5 commits
- [Model Runner V2] Enable piecewise CUDA graphs for pipeline parallelism (#35162) · 63f49b8b
  zhanqiuhu authored Mar 22, 2026
```
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
```
  63f49b8b
- [MRV2] Use FP64 for Gumbel noise (#37798) · a5e9d511
  Woosuk Kwon authored Mar 22, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  a5e9d511
- [MRV2] Skip hidden states allocation for PW CUDA graphs (#37818) · ce9b1d76
  Woosuk Kwon authored Mar 22, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  ce9b1d76
- [Model Runner V2] Support multi-modal embeddings for spec decode model (#36097) · b3e84601
  Giancarlo Delfin authored Mar 22, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
```
  b3e84601
- [Bugfix] Fix pooling non-determinism from pinned prompt_lens aliasing (#37775) · 66f927f2
  Andreas Karatzas authored Mar 21, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  66f927f2
21 Mar, 2026 1 commit
- [Hybrid] calling get_mamba_groups() once at MambaCopyBuffers.create() (#37318) · 298e5108
  Francesco Fusco authored Mar 21, 2026
```
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
```
  298e5108
20 Mar, 2026 7 commits
- elastic_ep: Fix issues with repeated scale up/down cycles (#37131) · c57d38d6
  Itay Alroy authored Mar 21, 2026
```
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
```
  c57d38d6
- [Model Runner V2] Support Streaming Inputs (#37028) · 85f671b8
  Santino Ramos authored Mar 20, 2026
```
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
```
  85f671b8
- [Attention] Support distinguishing between short extends and decodes (#37303) · e1d85e5c
  Lucas Wilkinson authored Mar 20, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  e1d85e5c
- fix CUDAGraph memory being counted twice (#37426) · 79eb9369
  Peter Pan authored Mar 21, 2026
```
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
```
  79eb9369
- [MRV2] Avoid recompilation of _gather_block_tables_kernel (#37645) · e80cfe57
  Woosuk Kwon authored Mar 20, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  e80cfe57
- [Model] Deprecate the score task (this will not affect users). (#37537) · ed359c49
  wang.yuqi authored Mar 20, 2026
```
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
```
  ed359c49
- [Model Runner V2] Fix draft logits not populated during cudagraph replay (#37639) · dcee9be9
  Giancarlo Delfin authored Mar 20, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  dcee9be9