Commits · 417fd28fb125cbb166ef3ada187d06d0c8dd0d30 · OpenDAS / vllm_cscc

04 Mar, 2026 4 commits
- [Model Runner V2] Fix pooling (#36019) · 417fd28f
  Nick Hill authored Mar 04, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  417fd28f
- [BugFix] Fix implicit and incorrect assumption on ECConnector is_producer (#34783) · 6aa6ad89
  Qi Wang authored Mar 04, 2026
```
Signed-off-by: Qi Wang <qiwa@nvidia.com>
```
  6aa6ad89
- [Bugfix] Cap FULL decode cudagraph sizes for Mamba/hybrid models (#34094) (#34571) · d6e04f4c
  haosdent authored Mar 04, 2026
```
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
```
  d6e04f4c
- [Hardware] Replace `torch.cuda.empty_cache` with `torch.accelerator.empty_cache` (#30681) · 16d2ad1d
  Kunshang Ji authored Mar 04, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  16d2ad1d
03 Mar, 2026 8 commits
- [Core] Move save_tensorized_model logic to Worker (#35825) · d15c3b90
  Nick Hill authored Mar 03, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  d15c3b90
- [Model Runner V2] support dp & ep for spec decoding (#35294) · 97286a20
  zhrrr authored Mar 04, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  97286a20
- [Model Runner V2] Fix inputs_embeds=None bug for MM models (#35917) · 467886a0
  Woosuk Kwon authored Mar 03, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  467886a0
- [BugFix] Add support for MTP num_speculative_tokens > 1 with sparse MLA (#34552) · 28ef9ba3
  Lucas Wilkinson authored Mar 03, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
```
  28ef9ba3
- [Perf] [Hybrid] Copy num_accepted_tokens in non-blocking way when not using prefix caching (#35442) · ad9d09e2
  Thomas Parnell authored Mar 03, 2026
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  ad9d09e2
- [Model Runner V2] Use ModelState.prepare_attn() for cuda graph capture [5/N] (#35774) · a0a5178a
  Woosuk Kwon authored Mar 02, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  a0a5178a
- [ModelRunnerV2] Rename sampler functions and variables for clarity (#35459) · 0a7165fd
  Andy Lo authored Mar 03, 2026
```
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  0a7165fd
- [Model Runner V2][Perf] align dummy_run tokens to uniform decode for dp cudagraph (#35376) · 168ee03e
  zhrrr authored Mar 03, 2026
```
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
```
  168ee03e
02 Mar, 2026 4 commits
- clean unused cudagraph_batch_sizes (#35552) · c42dc402
  Boyuan Feng authored Mar 02, 2026
```
Signed-off-by: Boyuan Feng <boyuan@meta.com>
```
  c42dc402
- [Spec Decode] Add hidden states extraction system (#33736) · 9433acb8
  Fynn Schmitt-Ulms authored Mar 02, 2026
```
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
```
  9433acb8
- [Fix Bug]`num_active_loras` always equals to zero (#34119) · ada4f4fa
  Runkai Tao authored Mar 02, 2026
```
Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ada4f4fa
- [MyPy][BugFix] Check profiler is assigned before calling start() on it (#35505) · 87c98b02
  Martin Hickey authored Mar 02, 2026
```
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  87c98b02
01 Mar, 2026 2 commits
- [Model Runner V2] Use block table apis for capture inputs (#35671) · 72f4d162
  Woosuk Kwon authored Mar 01, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  72f4d162
- [Model Runner V2] Minor refactoring for EncoderRunner (#35628) · da543d1a
  Woosuk Kwon authored Mar 01, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  da543d1a
28 Feb, 2026 6 commits

[Model Runner V2] Add ModelStateInterface [4/N] (#35621) · e3eb146f
Woosuk Kwon authored Feb 28, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
e3eb146f
[Bugfix] Propagate compilation_time from workers to main process for TP>1 (#35503) · 7b346ba8
Huy Do authored Feb 27, 2026
```
Signed-off-by: Huy Do <huydhn@gmail.com>
```
7b346ba8

[1/N] Elastic EP Milestone 2 (#34861) · dea26833

Itay Alroy authored Feb 28, 2026


Signed-off-by: Yongji Wu <wuyongji317@gmail.com>
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Ron Tourgeman <rtourgeman@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>

dea26833

[CI/Build] CPU release supports both of AVX2 and AVX512 (#35466) · 90805ff4
Ma Jian authored Feb 28, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
```
90805ff4
[Model Runner V2] Move MM encoder to Model States [3/N] (#35564) · 1a014a0a
Woosuk Kwon authored Feb 27, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
1a014a0a
[Model Runner V2] Support pooling models (#35120) · 86ac7bcf
Woosuk Kwon authored Feb 27, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
86ac7bcf

27 Feb, 2026 5 commits
- [DP] Only use DP padding when cudagraphs are actually used (#34102) · 1d532f9d
  Lucas Wilkinson authored Feb 27, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  1d532f9d
- [Model Runner V2] Warmup kernels (#35172) · b1d9f537
  Nick Hill authored Feb 27, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  b1d9f537
- [perf] Use pinned memory for async H2D transfer in do_mamba_copy_block (#35480) · 157722da
  Huamin Li authored Feb 27, 2026
```
Signed-off-by: Huamin Li <3ericli@gmail.com>
```
  157722da
- [Core] Fix `gpu_worker.py` pre-commit errors (#35312) · 876312f0
  Nick Hill authored Feb 27, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  876312f0
- [Misc] Move `GPUModelRunner.prepare_kernel_block_sizes` to utils (#35400) · cabdaa76
  Nicolò Lucchesi authored Feb 27, 2026
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  cabdaa76
26 Feb, 2026 6 commits
- [Model Runner V2] Prepare attn metadata in ModelState [2/N] (#35383) · 3d66502e
  Woosuk Kwon authored Feb 26, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  3d66502e
- [Model Runner V2] Add model states [1/N] (#35350) · c66aa48e
  Woosuk Kwon authored Feb 26, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  c66aa48e
- [Model Runner V2] Fix error-handling (#35063) · b6d5a172
  Nick Hill authored Feb 26, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  b6d5a172
- [Model Runner V2] Add coding style guide (#35325) · 13025e71
  Woosuk Kwon authored Feb 25, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  13025e71
- [Bugfix] Fix AttributeError in SMControlContextManager (#35338) · 9511a3f8
  Lucas Wilkinson authored Feb 25, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9511a3f8
- [offloader] v2: Hide weight onloading latency via prefetching (#29941) · 6831650c
  Ming Yang authored Feb 25, 2026
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  6831650c
25 Feb, 2026 3 commits

[XPU]Support CUDAGraph on XPU Platform (#34482) · 35d44b45

Xinyu Chen authored Feb 25, 2026


Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: chzhang <chaojun.zhang@intel.com>
Co-authored-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>

35d44b45

[Platform] Add current_platform.num_compute_units interface (#35042) · 8ad54a99
Kunshang Ji authored Feb 25, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
```
8ad54a99
[Linear Attention] fix bug for linear attention + prefix caching + reset_prefix_cache (#35157) · 8fae54fa
Chen Zhang authored Feb 24, 2026
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
8fae54fa

24 Feb, 2026 2 commits

[Model][Spec Decode] Nemotron-H MTP and Mamba Speculative Decoding Support (#33726) · f5972a87

Benjamin Chislett authored Feb 24, 2026


Signed-off-by: Shahar Mor <smor@nvidia.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Shahar Mor <smor@nvidia.com>
Co-authored-by: Roi Koren <roik@nvidia.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>

f5972a87

[Perf] Optimize pooling model redundant copy, 1.8% throughput improvement (#35127) · 14561fab
Wentao Ye authored Feb 24, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
14561fab