Commits · 2b7160c6d57dbb7ef7e7432d5652e6d9451aed84 · OpenDAS / vllm_cscc

23 Apr, 2026 1 commit
- vllm kvprune:v1.0.0 · 2b7160c6
  chenzk authored Apr 23, 2026
  
  2b7160c6
19 Mar, 2026 1 commit

[Bugfix][ROCm] Fix worker startup OOM on ROCm by skipping unreliable cudagraph... · 6edd43de

JartX authored Mar 17, 2026


[Bugfix][ROCm] Fix worker startup OOM on ROCm by skipping unreliable cudagraph memory profiling (#36720)
Signed-off-by: JartX <sagformas@epdcenter.es>
(cherry picked from commit e8f9dbc3)

6edd43de

16 Mar, 2026 6 commits
- [BUGFIX][Mamba] Use uint64 for address in KVBlockZeroer (#37197) · d1572160
  Kunshang Ji authored Mar 17, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  d1572160
- [Bugfix] Disable cross-layer KV cache for MLA attention backends (#37090) · ca1954d5
  haosdent authored Mar 17, 2026
```
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
```
  ca1954d5
- [Spec Decode] Update extract_hidden_states to use deferred kv_connector clear (#37013) · 04bf5a35
  Fynn Schmitt-Ulms authored Mar 16, 2026
  
  04bf5a35
- [Hardware] Replace memory related torch.cuda APIs (#37031) · 747b0681
  Kunshang Ji authored Mar 16, 2026
```
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
```
  747b0681
- [Model Runner V2] Fix processed logits in sample() (#37144) · 96efb914
  Woosuk Kwon authored Mar 16, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  96efb914
- [Bugfix] Avoid LD_PRELOAD check on MacOS (#37145) · 7362b445
  Li, Jiang authored Mar 16, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  7362b445
14 Mar, 2026 3 commits

[Model Runner V2] Add Support for XD-RoPE (#36817) · 3ed46f37
Santino Ramos authored Mar 14, 2026
```
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
```
3ed46f37
[V1] Remove pin_memory() in async_copy_to_gpu to fix sporadic stalls (#37006) · a116f969
sbeurnier authored Mar 14, 2026
```
Signed-off-by: Sebastien Beurnier <sbeurnier@together.ai>
```
a116f969

[UX] Improve UX of CPU backend (#36968) · 092ace9e

Li, Jiang authored Mar 14, 2026

Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Li, Jiang <bigpyj64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

092ace9e

13 Mar, 2026 3 commits
- [Refactor] Consolidate SupportsEagle (#36063) · 8b346309
  Benjamin Chislett authored Mar 13, 2026
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  8b346309
- [Bugfix][Spec Decode] Avoid double call of Ngram CPU (#36952) · d0b40297
  Ekagra Ranjan authored Mar 13, 2026
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
```
  d0b40297
- [Model Runner V2] Some code simplification (#36929) · cd32d6f5
  Nick Hill authored Mar 12, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  cd32d6f5
12 Mar, 2026 3 commits
- [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) · 53ec16a7
  Kunshang Ji authored Mar 12, 2026
```
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  53ec16a7
- [Model Runner V2] Do not initialize sampler for non-last PP ranks (#36824) · 2f8b4ce0
  Woosuk Kwon authored Mar 11, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  2f8b4ce0
- [Perf] Optimize compute maxsim using batched version, 3.2% E2E throughput improvement (#36710) · c34ba6b9
  Wentao Ye authored Mar 11, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  c34ba6b9
11 Mar, 2026 10 commits
- [BUG] Fix async rlhf tests (#35811) · d6b61e51
  Aaron Hao authored Mar 11, 2026
```
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
```
  d6b61e51
- [Model Runner V2] Add WhisperModelState [6/N] (#35790) · 55eed6b7
  Woosuk Kwon authored Mar 11, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  55eed6b7
- [Model Runner V2] Add probabilistic rejection sampling for spec decoding (#35461) · c77181e5
  Giancarlo Delfin authored Mar 11, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  c77181e5
- [KVConnector] Support worker -> scheduler metadata (#31964) · a1a3523a
  Or Ozeri authored Mar 11, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
```
  a1a3523a
- Fix routed experts capture for hybrid models (Mamba + Attention) (#35744) · bea02cdf
  Hongxin Xu authored Mar 11, 2026
```
Signed-off-by: arlenxu <arlenxu@tencent.com>
Signed-off-by: xhx1022 <1737006628@qq.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
```
  bea02cdf
- [Model Runner V2] Remove unused warmup_for_prefill method (#36762) · 8ccbcda5
  Woosuk Kwon authored Mar 11, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  8ccbcda5
- [XPU]Bug fix for some unexpected error when use AgRs backend on XPU device. (#36593) · c910eeb1
  YiSheng5 authored Mar 11, 2026
```
Signed-off-by: yisheng <yi.sheng@intel.com>
```
  c910eeb1
- [NIXL][1/N] Refactor `kernel_block_size` detection (#35752) · 098d8447
  Nicolò Lucchesi authored Mar 11, 2026
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  098d8447
- [Doc] Fix duplicate words in comments (#36713) · 4bf53362
  Hongbin Guo authored Mar 11, 2026
```
Signed-off-by: Hongbin10 <jdmjdm1998@163.com>
```
  4bf53362
- [Bugfix] Fix DeepSeek V3.2 OOM during CG memory profiling (#36691) · 8ab3d742
  Matthew Bonanni authored Mar 10, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  8ab3d742
10 Mar, 2026 6 commits
- [Core] Simplify core kv-cache blocks initialization logic (#36521) · 65b2f405
  Nick Hill authored Mar 10, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  65b2f405
- [Model Runner V2] Use unpadded num_tokens for PW CUDA graph attn metadata (#36626) · f088a831
  Woosuk Kwon authored Mar 10, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  f088a831
- [BUGFIX][Mamba][Qwen3.5] Zero freed SSM cache blocks on GPU (#35219) · 4ff8c3c8
  Vadim Gimpelson authored Mar 10, 2026
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
```
  4ff8c3c8
- [Model Runner V2] Fix mm input embeddings lookup (#36588) · ddbb0d23
  Nick Hill authored Mar 10, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  ddbb0d23
- [Model Runner V2] Fix `_compute_slot_mappings_kernel` for chunked prefill (#36580) · 9efc3bdc
  Nick Hill authored Mar 10, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  9efc3bdc
- [Perf] Compute maxsim in worker side, reducing redundant copies, 2.7% E2E... · 7279374f
  Wentao Ye authored Mar 09, 2026
```
[Perf] Compute maxsim in worker side, reducing redundant copies, 2.7% E2E throughput improvement (#36159)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  7279374f
09 Mar, 2026 7 commits
- [Model Runner V2] Add model_state inputs to CUDA graph capture (#36544) · 2a194ddd
  Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  2a194ddd
- [MRV2] Extensible CG dispatch rework (#35959) · 483463f7
  Lucas Wilkinson authored Mar 09, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  483463f7
- [Model Runner V2] Use NamedTuple for `execute_model_state` (#35930) · 10a5f4d5
  Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  10a5f4d5
- [Model Runner V2] Add dummy profile_cudagraph_memory API (#36520) · 6e956d9e
  Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  6e956d9e
- [Bugfix] Clear stale CG keys after memory profiling (#36416) · 00c4cb56
  Matthew Bonanni authored Mar 09, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  00c4cb56
- Reapply [Attention] Refactor `check_and_update_config` (#35122) · 77a73458
  Matthew Bonanni authored Mar 09, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  77a73458
- [Bugfix] Avoid to replace non-tensor members in cpu model runner (#36430) · 217f2759
  Li, Jiang authored Mar 09, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  217f2759