Commits · 71dfce6aa6cc14d016154b4e3fd8cc40c05415f9 · OpenDAS / vllm_cscc

26 Feb, 2026 1 commit

[offloader] v2: Hide weight onloading latency via prefetching (#29941) · 6831650c

Ming Yang authored Feb 25, 2026


Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

6831650c

23 Feb, 2026 2 commits
- [Model Runner V2] Remove propose_draft method (#35070) · c645e9a2
  Woosuk Kwon authored Feb 22, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  c645e9a2
- [Model Runner V2][Minor] Remove redundant `do_spec_decode` field (#35039) · 944ffb59
  Nick Hill authored Feb 22, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
```
  944ffb59
22 Feb, 2026 3 commits

[Spec Decode] Defer clearing KV connector metadata for EAGLE3 speculative... · b9c2a565

qizixi authored Feb 22, 2026


[Spec Decode] Defer clearing KV connector metadata for EAGLE3 speculative decode + prefill / decode disagg setup (#34529)
Signed-off-by: qizixi <qizixi@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>

b9c2a565

[Model Runner V2] Enable CUDA graph for Eagle3 (#35040) · 2cbf9656
Woosuk Kwon authored Feb 21, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
2cbf9656
[Model Runner V2] Support attention group (#35036) · b71fbd06
Woosuk Kwon authored Feb 21, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
b71fbd06

21 Feb, 2026 1 commit
- [Model Runner V2] Support Eagle3 (no CUDA graph) (#35029) · a4047d4e
  Woosuk Kwon authored Feb 21, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  a4047d4e
20 Feb, 2026 1 commit
- [Model Runner V2] Minor CPU optimizations (#34856) · 40b2f1c3
  Nick Hill authored Feb 19, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  40b2f1c3
19 Feb, 2026 1 commit
- [Model Runner V2] Use FP32 for Gumbel Noise (#34854) · 5fcb0cdd
  Woosuk Kwon authored Feb 18, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  5fcb0cdd
18 Feb, 2026 6 commits
- [Model Runner V2] Remove unnecessary copies in PW CUDA graph capture (#34849) · c878b43b
  Woosuk Kwon authored Feb 18, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  c878b43b
- [Model Runner V2] support piecewise & mixed cudagraph (#32771) · 11d3976b
  zhrrr authored Feb 19, 2026
```
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
```
  11d3976b
- [Model Runner V2] Minor simplification for DCP (#34786) · 95be2a7f
  Woosuk Kwon authored Feb 18, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  95be2a7f
- [Model Runner V2] Avoid prepare prefill kernel launch overhead (#34780) · c50e105a
  Nick Hill authored Feb 18, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  c50e105a
- [Model Runner V2] A bit more PP simplification (#34766) · a49ea5a5
  Nick Hill authored Feb 17, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  a49ea5a5
- [Feature] Decode Context Parallel support for GPU model runner v2 (#34179) · ab33d2a6
  Wentao Ye authored Feb 17, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  ab33d2a6
17 Feb, 2026 8 commits
- [Model Runner V2] Further simplification for PP (#34724) · be3af2d2
  Woosuk Kwon authored Feb 17, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  be3af2d2
- Fix docs build warning (#34686) · 28bffe94
  Harry Mellor authored Feb 17, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  28bffe94
- [Model Runner V2] Minor refactoring for penalties (#34662) · d00df624
  Woosuk Kwon authored Feb 16, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  d00df624
- [Model Runner V2] Minor simplification for BadWordsState (#34669) · 9752da9d
  Woosuk Kwon authored Feb 16, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  9752da9d
- [Model Runner V2] Minor cleanup for PP (#34666) · 04925b22
  Woosuk Kwon authored Feb 16, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  04925b22
- [Model Runner V2] Fix unintended CPU-GPU sync in make_dummy (#34667) · d74278fb
  Woosuk Kwon authored Feb 16, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  d74278fb
- [Core] Pipeline Parallel support for Model Runner V2 (#33960) · 9a8853f7
  zhanqiuhu authored Feb 16, 2026
```
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
```
  9a8853f7
- [Model Runner V2] support bad_words sampling param (#33433) · 387a1898
  zhrrr authored Feb 17, 2026
```
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
```
  387a1898
15 Feb, 2026 1 commit
- [Model Runner V2] Minor cleanup for Sampler (#34563) · 9ca768c7
  Woosuk Kwon authored Feb 14, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  9ca768c7
11 Feb, 2026 1 commit
- [Model Runner V2] Init cuda graph pool when necessary (#33217) · ffb3d553
  Xinyu Chen authored Feb 12, 2026
```
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
```
  ffb3d553
10 Feb, 2026 1 commit
- [Model Runner V2] Use pinned memory for write_contents (#34222) · a2443de5
  Woosuk Kwon authored Feb 10, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  a2443de5
09 Feb, 2026 1 commit
- [ModelRunner V2][BugFix] Fix `max_query_len` calculation (#34167) · e7e52781
  Nick Hill authored Feb 09, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  e7e52781
07 Feb, 2026 1 commit
- [ModelRunner V2] Revert token rank comparison difference for now (#34017) · 40218a82
  Nick Hill authored Feb 06, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  40218a82
06 Feb, 2026 2 commits
- [Model Runner V2] support apply penalty for spec decode (#33251) · 16786da7
  zhrrr authored Feb 07, 2026
```
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
```
  16786da7
- support view_from_cpu_tensor on XPU (#33868) · e969a169
  Xinyu Chen authored Feb 06, 2026
```
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
```
  e969a169
04 Feb, 2026 1 commit
- [Core] Don't schedule spec tokens with prefill chunks (#33652) · fa4e0fb0
  Nick Hill authored Feb 04, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  fa4e0fb0
02 Feb, 2026 1 commit
- [ModelRunner V2] Support spec decode with structured outputs (#33374) · cf0a99f8
  Nick Hill authored Feb 01, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  cf0a99f8
01 Feb, 2026 2 commits
- [ModelRunner V2] Misc minor simplifications and optimizations (#33467) · e535d90d
  Nick Hill authored Feb 01, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  e535d90d
- [Bugfix] Fix inconsistent handling of cache reset (#33481) · 79b6ec6a
  Cyrus Leung authored Feb 01, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  79b6ec6a
31 Jan, 2026 1 commit
- [ModelRunner V2] Fix spec decoding + logprobs (#33391) · 876a16f4
  Nick Hill authored Jan 30, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  876a16f4
29 Jan, 2026 1 commit
- [Multimodal] Simplify MM input definitions (#33331) · c6e7404c
  Cyrus Leung authored Jan 29, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c6e7404c
28 Jan, 2026 2 commits
- [ModelRunner V2] Misc code simplification and cleanup (#33266) · 6bf3b46d
  Nick Hill authored Jan 28, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  6bf3b46d
- [Model Runner V2] Use a different stream for grammar bitmask h2d copy (#33059) · 46ec6d71
  Woosuk Kwon authored Jan 27, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  46ec6d71
27 Jan, 2026 1 commit

[Model Runner V2] Remove UvaBufferPool for cpu->gpu copy (#33055) · 6d86fde0

Woosuk Kwon authored Jan 26, 2026


Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Nick Hill <nhill@redhat.com>

6d86fde0

26 Jan, 2026 1 commit
- [Model Runner V2] Add LoRAState to consolidate lora logic (#33062) · a9b53dd4
  Woosuk Kwon authored Jan 25, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  a9b53dd4