Commits · 2ef69456f5a0078bd8a8614b1cb6376e730e4d20 · OpenDAS / vllm_cscc

12 Mar, 2026 1 commit
- [Perf] Optimize compute maxsim using batched version, 3.2% E2E throughput improvement (#36710) · c34ba6b9
  Wentao Ye authored Mar 11, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  c34ba6b9
11 Mar, 2026 3 commits
- [Model Runner V2] Add WhisperModelState [6/N] (#35790) · 55eed6b7
  Woosuk Kwon authored Mar 11, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  55eed6b7
- [Model Runner V2] Add probabilistic rejection sampling for spec decoding (#35461) · c77181e5
  Giancarlo Delfin authored Mar 11, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  c77181e5
- [Model Runner V2] Remove unused warmup_for_prefill method (#36762) · 8ccbcda5
  Woosuk Kwon authored Mar 11, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  8ccbcda5
10 Mar, 2026 4 commits
- [Model Runner V2] Use unpadded num_tokens for PW CUDA graph attn metadata (#36626) · f088a831
  Woosuk Kwon authored Mar 10, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  f088a831
- [Model Runner V2] Fix mm input embeddings lookup (#36588) · ddbb0d23
  Nick Hill authored Mar 10, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  ddbb0d23
- [Model Runner V2] Fix `_compute_slot_mappings_kernel` for chunked prefill (#36580) · 9efc3bdc
  Nick Hill authored Mar 10, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  9efc3bdc
- [Perf] Compute maxsim in worker side, reducing redundant copies, 2.7% E2E... · 7279374f
  Wentao Ye authored Mar 09, 2026
```
[Perf] Compute maxsim in worker side, reducing redundant copies, 2.7% E2E throughput improvement (#36159)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  7279374f
09 Mar, 2026 4 commits
- [Model Runner V2] Add model_state inputs to CUDA graph capture (#36544) · 2a194ddd
  Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  2a194ddd
- [MRV2] Extensible CG dispatch rework (#35959) · 483463f7
  Lucas Wilkinson authored Mar 09, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  483463f7
- [Model Runner V2] Use NamedTuple for `execute_model_state` (#35930) · 10a5f4d5
  Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  10a5f4d5
- [Model Runner V2] Add dummy profile_cudagraph_memory API (#36520) · 6e956d9e
  Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  6e956d9e
07 Mar, 2026 1 commit
- [Model Runner V2] Fix warmup for pipeline parallel (#36280) · b3546865
  Nick Hill authored Mar 06, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  b3546865
06 Mar, 2026 1 commit
- [Misc] Rename `group_mm_kwargs_by_modality -> group_and_batch_mm_kwargs` (#36158) · 6dd30265
  Cyrus Leung authored Mar 06, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6dd30265
05 Mar, 2026 2 commits
- [Model Runner V2] Fix warmup for very small kvcache and/or blocksizes (#36176) · a73af584
  Nick Hill authored Mar 05, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  a73af584
- [Hardware] Replace `torch.cuda.synchronize()` api with `torch.accelerator.synchronize` (#36085) · 66a22096
  Kunshang Ji authored Mar 05, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  66a22096
04 Mar, 2026 3 commits
- [Model Runner V2] Misc code simplification (#35941) · a3299c3d
  Nick Hill authored Mar 04, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  a3299c3d
- [Model Runner V2] Fix pooling (#36019) · 417fd28f
  Nick Hill authored Mar 04, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  417fd28f
- [Hardware] Replace `torch.cuda.empty_cache` with `torch.accelerator.empty_cache` (#30681) · 16d2ad1d
  Kunshang Ji authored Mar 04, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  16d2ad1d
03 Mar, 2026 5 commits
- [Model Runner V2] support dp & ep for spec decoding (#35294) · 97286a20
  zhrrr authored Mar 04, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  97286a20
- [Model Runner V2] Fix inputs_embeds=None bug for MM models (#35917) · 467886a0
  Woosuk Kwon authored Mar 03, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  467886a0
- [Model Runner V2] Use ModelState.prepare_attn() for cuda graph capture [5/N] (#35774) · a0a5178a
  Woosuk Kwon authored Mar 02, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  a0a5178a
- [ModelRunnerV2] Rename sampler functions and variables for clarity (#35459) · 0a7165fd
  Andy Lo authored Mar 03, 2026
```
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  0a7165fd
- [Model Runner V2][Perf] align dummy_run tokens to uniform decode for dp cudagraph (#35376) · 168ee03e
  zhrrr authored Mar 03, 2026
```
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
```
  168ee03e
01 Mar, 2026 2 commits
- [Model Runner V2] Use block table apis for capture inputs (#35671) · 72f4d162
  Woosuk Kwon authored Mar 01, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  72f4d162
- [Model Runner V2] Minor refactoring for EncoderRunner (#35628) · da543d1a
  Woosuk Kwon authored Mar 01, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  da543d1a
28 Feb, 2026 3 commits
- [Model Runner V2] Add ModelStateInterface [4/N] (#35621) · e3eb146f
  Woosuk Kwon authored Feb 28, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  e3eb146f
- [Model Runner V2] Move MM encoder to Model States [3/N] (#35564) · 1a014a0a
  Woosuk Kwon authored Feb 27, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  1a014a0a
- [Model Runner V2] Support pooling models (#35120) · 86ac7bcf
  Woosuk Kwon authored Feb 27, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  86ac7bcf
27 Feb, 2026 1 commit
- [Model Runner V2] Warmup kernels (#35172) · b1d9f537
  Nick Hill authored Feb 27, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  b1d9f537
26 Feb, 2026 5 commits
- [Model Runner V2] Prepare attn metadata in ModelState [2/N] (#35383) · 3d66502e
  Woosuk Kwon authored Feb 26, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  3d66502e
- [Model Runner V2] Add model states [1/N] (#35350) · c66aa48e
  Woosuk Kwon authored Feb 26, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  c66aa48e
- [Model Runner V2] Fix error-handling (#35063) · b6d5a172
  Nick Hill authored Feb 26, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  b6d5a172
- [Model Runner V2] Add coding style guide (#35325) · 13025e71
  Woosuk Kwon authored Feb 25, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  13025e71
- [offloader] v2: Hide weight onloading latency via prefetching (#29941) · 6831650c
  Ming Yang authored Feb 25, 2026
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  6831650c
23 Feb, 2026 2 commits
- [Model Runner V2] Remove propose_draft method (#35070) · c645e9a2
  Woosuk Kwon authored Feb 22, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  c645e9a2
- [Model Runner V2][Minor] Remove redundant `do_spec_decode` field (#35039) · 944ffb59
  Nick Hill authored Feb 22, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
```
  944ffb59
22 Feb, 2026 3 commits

[Spec Decode] Defer clearing KV connector metadata for EAGLE3 speculative... · b9c2a565

qizixi authored Feb 22, 2026


[Spec Decode] Defer clearing KV connector metadata for EAGLE3 speculative decode + prefill / decode disagg setup (#34529)
Signed-off-by: qizixi <qizixi@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>

b9c2a565

[Model Runner V2] Enable CUDA graph for Eagle3 (#35040) · 2cbf9656
Woosuk Kwon authored Feb 21, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
2cbf9656
[Model Runner V2] Support attention group (#35036) · b71fbd06
Woosuk Kwon authored Feb 21, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
b71fbd06