Commits · 4d51588e2381018348f1022dfa3a7698899805b7 · OpenDAS / vllm_cscc · GitLab

27 Apr, 2026 1 commit

[Feat] DeepSeek V4 Rebased (#40860) · 4d51588e

Yifan Qiao authored Apr 26, 2026


Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>

4d51588e

24 Apr, 2026 2 commits

[Feat] Unified Synthetic Acceptance Rate for V1 and V2 (#40662) · 626daa20

Benjamin Chislett authored Apr 23, 2026

Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

626daa20

[Core] Avoid seq_lens_cpu GPU->CPU sync (#40654) · fe85a92e
Nick Hill authored Apr 23, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
fe85a92e

16 Apr, 2026 1 commit
- Gate SSU dispatch setup (#40039) · b897f00c
  roikoren755 authored Apr 16, 2026
```
Signed-off-by: Roi Koren <roik@nvidia.com>
```
  b897f00c
14 Apr, 2026 1 commit
- [Mamba] Flashinfer selective_state_update (#36162) · ecf5ff7c
  roikoren755 authored Apr 14, 2026
```
Signed-off-by: Roi Koren <roik@nvidia.com>
```
  ecf5ff7c
13 Apr, 2026 1 commit
- [Model Runner V2] Add full cuda graph support for eagle prefill (#37588) · ccf90ba7
  Giancarlo Delfin authored Apr 13, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  ccf90ba7
10 Apr, 2026 1 commit
- [Model Runner V2] support auto resolve cudagraph mode/sizes based on attn backend (#32936) · 8f121f78
  zhrrr authored Apr 10, 2026
```
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
```
  8f121f78
07 Apr, 2026 1 commit
- [BugFix][MRV2] Fix cuda event reuse race (#39115) · 62095e82
  Nick Hill authored Apr 06, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  62095e82
06 Apr, 2026 1 commit
- [MRV2] Fix hanging issue with DeepSeek V3.2 by setting `skip_attn=False` (#39098) · f186cfe7
  Woosuk Kwon authored Apr 06, 2026
```
Signed-off-by: WoosukKwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  f186cfe7
26 Mar, 2026 1 commit
- [Model Runner V2] Enable forcing a specific acceptance rate during rejection sampling (#38045) · c32e9760
  Giancarlo Delfin authored Mar 26, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  c32e9760
25 Mar, 2026 1 commit

[Feature] EPLB Support for GPU Model Runner v2 (#37488) · d7e93e13

Wentao Ye authored Mar 25, 2026


Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>

d7e93e13

24 Mar, 2026 2 commits
- [Model Runner V2][Minor] Simplify PP logic (#38031) · 4e824d1c
  Nick Hill authored Mar 24, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  4e824d1c
- [Model Runner V2] Gather multimodal embeddings before draft model postprocess (#37932) · 8f4824b6
  Giancarlo Delfin authored Mar 23, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  8f4824b6
22 Mar, 2026 2 commits

[Model Runner V2] Enable piecewise CUDA graphs for pipeline parallelism (#35162) · 63f49b8b

zhanqiuhu authored Mar 22, 2026


Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>

63f49b8b

[Model Runner V2] Support multi-modal embeddings for spec decode model (#36097) · b3e84601

Giancarlo Delfin authored Mar 22, 2026


Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>

b3e84601

20 Mar, 2026 3 commits
- [Model Runner V2] Support Streaming Inputs (#37028) · 85f671b8
  Santino Ramos authored Mar 20, 2026
```
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
```
  85f671b8
- [Model Runner V2] Fix draft logits not populated during cudagraph replay (#37639) · dcee9be9
  Giancarlo Delfin authored Mar 20, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  dcee9be9
- [Model Runner V2] fix draft attention metadata generation (#37364) · 39474513
  Giancarlo Delfin authored Mar 19, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  39474513
19 Mar, 2026 1 commit
- [MRV2] Use fp32 for draft logits (#37526) · 40b8363b
  Woosuk Kwon authored Mar 19, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  40b8363b
18 Mar, 2026 1 commit
- [Model Runner V2] Spec decode rejection sampler greedy support (#37238) · 04244fd0
  Giancarlo Delfin authored Mar 18, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  04244fd0
14 Mar, 2026 1 commit
- [Model Runner V2] Add Support for XD-RoPE (#36817) · 3ed46f37
  Santino Ramos authored Mar 14, 2026
```
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
```
  3ed46f37
12 Mar, 2026 1 commit
- [Model Runner V2] Do not initialize sampler for non-last PP ranks (#36824) · 2f8b4ce0
  Woosuk Kwon authored Mar 11, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  2f8b4ce0
11 Mar, 2026 3 commits
- [Model Runner V2] Add WhisperModelState [6/N] (#35790) · 55eed6b7
  Woosuk Kwon authored Mar 11, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  55eed6b7
- [Model Runner V2] Add probabilistic rejection sampling for spec decoding (#35461) · c77181e5
  Giancarlo Delfin authored Mar 11, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  c77181e5
- [Model Runner V2] Remove unused warmup_for_prefill method (#36762) · 8ccbcda5
  Woosuk Kwon authored Mar 11, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  8ccbcda5
10 Mar, 2026 1 commit
- [Model Runner V2] Use unpadded num_tokens for PW CUDA graph attn metadata (#36626) · f088a831
  Woosuk Kwon authored Mar 10, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  f088a831
09 Mar, 2026 3 commits
- [MRV2] Extensible CG dispatch rework (#35959) · 483463f7
  Lucas Wilkinson authored Mar 09, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  483463f7
- [Model Runner V2] Use NamedTuple for `execute_model_state` (#35930) · 10a5f4d5
  Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  10a5f4d5
- [Model Runner V2] Add dummy profile_cudagraph_memory API (#36520) · 6e956d9e
  Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  6e956d9e
05 Mar, 2026 1 commit
- [Hardware] Replace `torch.cuda.synchronize()` api with `torch.accelerator.synchronize` (#36085) · 66a22096
  Kunshang Ji authored Mar 05, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  66a22096
04 Mar, 2026 3 commits
- [Model Runner V2] Misc code simplification (#35941) · a3299c3d
  Nick Hill authored Mar 04, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  a3299c3d
- [Model Runner V2] Fix pooling (#36019) · 417fd28f
  Nick Hill authored Mar 04, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  417fd28f
- [Hardware] Replace `torch.cuda.empty_cache` with `torch.accelerator.empty_cache` (#30681) · 16d2ad1d
  Kunshang Ji authored Mar 04, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  16d2ad1d
03 Mar, 2026 4 commits
- [Model Runner V2] support dp & ep for spec decoding (#35294) · 97286a20
  zhrrr authored Mar 04, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  97286a20
- [Model Runner V2] Fix inputs_embeds=None bug for MM models (#35917) · 467886a0
  Woosuk Kwon authored Mar 03, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  467886a0
- [Model Runner V2] Use ModelState.prepare_attn() for cuda graph capture [5/N] (#35774) · a0a5178a
  Woosuk Kwon authored Mar 02, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  a0a5178a
- [Model Runner V2][Perf] align dummy_run tokens to uniform decode for dp cudagraph (#35376) · 168ee03e
  zhrrr authored Mar 03, 2026
```
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
```
  168ee03e
28 Feb, 2026 3 commits
- [Model Runner V2] Add ModelStateInterface [4/N] (#35621) · e3eb146f
  Woosuk Kwon authored Feb 28, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  e3eb146f
- [Model Runner V2] Move MM encoder to Model States [3/N] (#35564) · 1a014a0a
  Woosuk Kwon authored Feb 27, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  1a014a0a
- [Model Runner V2] Support pooling models (#35120) · 86ac7bcf
  Woosuk Kwon authored Feb 27, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  86ac7bcf