Commits · 417fd28fb125cbb166ef3ada187d06d0c8dd0d30 · OpenDAS / vllm_cscc

04 Mar, 2026 11 commits
- [Model Runner V2] Fix pooling (#36019) · 417fd28f
  Nick Hill authored Mar 04, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  417fd28f
- [Frontend] Add vllm launch command for GPU-less preprocessing serving (#34551) · bc6be89d
  Hyunkyun Moon authored Mar 05, 2026
```
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com>
```
  bc6be89d
- [Core] Add All-to-All communication backend for DCP (#34883) · 6cb90109
  sungsoo ha authored Mar 04, 2026
```
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com>
Signed-off-by: sungsoo ha <hasungsoo@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6cb90109
- [BugFix] Fix implicit and incorrect assumption on ECConnector is_producer (#34783) · 6aa6ad89
  Qi Wang authored Mar 04, 2026
```
Signed-off-by: Qi Wang <qiwa@nvidia.com>
```
  6aa6ad89
- [Bugfix][CPUOffloadingManager] Prevent eviction of already-stored blocks in... · bb6888b8
  Ronen Schaffer authored Mar 04, 2026
```
[Bugfix][CPUOffloadingManager] Prevent eviction of already-stored blocks in LRU/ARC `prepare_store()` (#35846)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
```
  bb6888b8
- [Bugfix] Cap FULL decode cudagraph sizes for Mamba/hybrid models (#34094) (#34571) · d6e04f4c
  haosdent authored Mar 04, 2026
```
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
```
  d6e04f4c
- [Hardware] Replace `torch.cuda.empty_cache` with `torch.accelerator.empty_cache` (#30681) · 16d2ad1d
  Kunshang Ji authored Mar 04, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  16d2ad1d
- [Core] Remove busy loop from idle buffer readers (#28053) · 6f0dd938
  Joe Runde authored Mar 04, 2026
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
```
  6f0dd938
- [cohere][fix][spec-decode]: fix crash when allowed_token_ids is set without penalties (#35654) · 9e0f44be
  Komal Kumar Teru authored Mar 04, 2026
```
Signed-off-by: kkt-cohere <komal@cohere.com>
```
  9e0f44be
- [Bugfix] Improve engine ready timeout error message (#35616) · 097eb544
  lailoo authored Mar 04, 2026
```
Signed-off-by: damaozi <1811866786@qq.com>
```
  097eb544
- [Bugfix] Fix coord_socket assertion in DPEngineCoreProc for offline DP mode (#35916) · f22ff295
  Jaewon authored Mar 03, 2026
```
Signed-off-by: Jaewon Lee <jaewon@meta.com>
```
  f22ff295
03 Mar, 2026 9 commits
- [Core] Move save_tensorized_model logic to Worker (#35825) · d15c3b90
  Nick Hill authored Mar 03, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  d15c3b90
- [Model Runner V2] support dp & ep for spec decoding (#35294) · 97286a20
  zhrrr authored Mar 04, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  97286a20
- [Model Runner V2] Fix inputs_embeds=None bug for MM models (#35917) · 467886a0
  Woosuk Kwon authored Mar 03, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  467886a0
- [BugFix] Add support for MTP num_speculative_tokens > 1 with sparse MLA (#34552) · 28ef9ba3
  Lucas Wilkinson authored Mar 03, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
```
  28ef9ba3
- [Perf] [Hybrid] Copy num_accepted_tokens in non-blocking way when not using prefix caching (#35442) · ad9d09e2
  Thomas Parnell authored Mar 03, 2026
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  ad9d09e2
- [Core] Add optional flags to check for repetitive token patterns in engine output (#35451) · 25e02647
  aykoppol authored Mar 02, 2026
```
Signed-off-by: aykoppol <aykoppol+git@gmail.com>
```
  25e02647
- [Model Runner V2] Use ModelState.prepare_attn() for cuda graph capture [5/N] (#35774) · a0a5178a
  Woosuk Kwon authored Mar 02, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  a0a5178a
- [ModelRunnerV2] Rename sampler functions and variables for clarity (#35459) · 0a7165fd
  Andy Lo authored Mar 03, 2026
```
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  0a7165fd
- [Model Runner V2][Perf] align dummy_run tokens to uniform decode for dp cudagraph (#35376) · 168ee03e
  zhrrr authored Mar 03, 2026
```
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
```
  168ee03e
02 Mar, 2026 6 commits
- clean unused cudagraph_batch_sizes (#35552) · c42dc402
  Boyuan Feng authored Mar 02, 2026
```
Signed-off-by: Boyuan Feng <boyuan@meta.com>
```
  c42dc402
- [Spec Decode] Add hidden states extraction system (#33736) · 9433acb8
  Fynn Schmitt-Ulms authored Mar 02, 2026
```
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
```
  9433acb8
- [Performance] Extract kv update ops from MLA attention backends (#34627) · d9c77308
  ElizaWszola authored Mar 02, 2026
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Di Wu <dw2761@nyu.edu>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  d9c77308
- [Fix Bug]`num_active_loras` always equals to zero (#34119) · ada4f4fa
  Runkai Tao authored Mar 02, 2026
```
Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ada4f4fa
- [MyPy][BugFix] Check profiler is assigned before calling start() on it (#35505) · 87c98b02
  Martin Hickey authored Mar 02, 2026
```
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  87c98b02
- [Misc] Cleanup useless `current_platform` import (#35715) · 510bc9e1
  wangxiyuan authored Mar 02, 2026
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  510bc9e1
01 Mar, 2026 5 commits
- [Attention] FA4 integration (#32974) · 8b5014d3
  Lucas Wilkinson authored Mar 01, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  8b5014d3
- Revert "[Bugfix] Disable TRTLLM attention with KV transfer enabled (#33192)" (#34832) · 57a96e26
  zhanqiuhu authored Mar 01, 2026
```
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
```
  57a96e26
- [Model Runner V2] Use block table apis for capture inputs (#35671) · 72f4d162
  Woosuk Kwon authored Mar 01, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  72f4d162
- [Mamba1] - Kernel Level Chunk Alignment for Prefix Caching (#34798) · bbf81f9a
  Asaf Gardin authored Mar 01, 2026
```
Signed-off-by: Josephasafg <ajgard7@gmail.com>
```
  bbf81f9a
- [Model Runner V2] Minor refactoring for EncoderRunner (#35628) · da543d1a
  Woosuk Kwon authored Mar 01, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  da543d1a
28 Feb, 2026 9 commits

[Model Runner V2] Add ModelStateInterface [4/N] (#35621) · e3eb146f
Woosuk Kwon authored Feb 28, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
e3eb146f

[Deprecation] Deprecate code in 0.17 as scheduled (#35441) · e113a301

Wentao Ye authored Feb 28, 2026


Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

e113a301

[Fix] Avoid sending image input to other PP ranks (#35405) · 49b9ae32

emricksini-h authored Feb 28, 2026


Signed-off-by: emricksini-h <emrick.birivoutin@hcompany.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>

49b9ae32

[Feat] Add CUDA torch fallbacks for fp8_mqa_logits/fp8_paged_mqa_logits_torch function (#35271) · 7e08c22b
Chauncey authored Feb 28, 2026
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
7e08c22b

add io_process_plugin for sparse embedding (#34214) · 8e75d885

Augusto Yao authored Feb 28, 2026


Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>
Signed-off-by: Augusto Yao <augusto.yjh@antgroup.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

8e75d885

[Bugfix] Propagate compilation_time from workers to main process for TP>1 (#35503) · 7b346ba8
Huy Do authored Feb 27, 2026
```
Signed-off-by: Huy Do <huydhn@gmail.com>
```
7b346ba8

[1/N] Elastic EP Milestone 2 (#34861) · dea26833

Itay Alroy authored Feb 28, 2026


Signed-off-by: Yongji Wu <wuyongji317@gmail.com>
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Ron Tourgeman <rtourgeman@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>

dea26833

[CI/Build] CPU release supports both of AVX2 and AVX512 (#35466) · 90805ff4
Ma Jian authored Feb 28, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
```
90805ff4
[ROCm] Add `stablelm` Head Size 80 To Supported Head Sizes For ROCM_ATTN (#35527) · 0edf101d
Micah Williamson authored Feb 27, 2026
```
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
```
0edf101d