Commits · 0231ce836a27ad0ed722e4b7162821baeeb19fdf · OpenDAS / vllm_cscc

25 Nov, 2025 3 commits

Revert back to torch.equal over torch.allclose from #28819 (#29086) · 0231ce83
Eldar Kurtić authored Nov 25, 2025
```
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
```
0231ce83

Rémi Delacourt authored Nov 25, 2025


Signed-off-by: Rémi Delacourt <remi@mistral.ai>
Signed-off-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Signed-off-by: remi <remi@mistral.ai>

12c007e2

[Spec Decode] Add support for EAGLE3 heads that do not use_aux_hidden_states (#27688) · 5f9679a4

Hanjie Qiu authored Nov 24, 2025


Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>

5f9679a4

22 Nov, 2025 1 commit
- [BugFix] Fix Eagle `IndexError: list index out of range` for even `num_speculative_tokens` (#29102) · 30d64662
  Lucas Wilkinson authored Nov 21, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  30d64662
21 Nov, 2025 1 commit
- Revert "[Redo] #26368 (#28771)" (#29121) · 30b9c677
  Jialin Ouyang authored Nov 20, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  30b9c677
20 Nov, 2025 1 commit
- [Bugfix] Fix spec decode memory regression after #28549 (#28819) · 93c8672c
  Zhewen Li authored Nov 20, 2025
```
Signed-off-by: zhewenli <zhewenli@meta.com>
```
  93c8672c
19 Nov, 2025 1 commit

Eagle: MM Cuda Graphs with MRope (#28896) · 02f5903b

Izzy Putterman authored Nov 19, 2025


Signed-off-by: Izzy Putterman <iputterman@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

02f5903b

17 Nov, 2025 1 commit

[Core] Async Scheduling X Spec Decoding Compatibility (#24799) · d8874c61

Ronald authored Nov 18, 2025


Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>

d8874c61

15 Nov, 2025 4 commits
- Add support for Eagle with separate lm-head and embed_tokens layers (#28549) · e439c784
  Eldar Kurtić authored Nov 15, 2025
```
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
```
  e439c784
- [Redo] #26368 (#28771) · 98b4d389
  Cyrus Leung authored Nov 15, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  98b4d389
- Revert "[Core] Performance: Use list[np.ndarray] instead of list[list… (#28773) · ac86bff8
  Nick Hill authored Nov 14, 2025
  
  ac86bff8
- [Core] Performance: Use list[np.ndarray] instead of list[list[int]] for output... · 186352b2
  Jialin Ouyang authored Nov 14, 2025
```
[Core] Performance: Use list[np.ndarray] instead of list[list[int]] for output tokens for GC optimization (#26368)
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  186352b2
13 Nov, 2025 1 commit
- Rename clashing method names for vLLM model protocol (#27583) · 97d1c993
  Harry Mellor authored Nov 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  97d1c993
11 Nov, 2025 1 commit

[Attention] Refactor CUDA attention backend selection logic (#24794) · b30dfa03

Matthew Bonanni authored Nov 11, 2025


Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

b30dfa03

04 Nov, 2025 1 commit

[model] Add support for openPangu_Ultra_MoE (#27521) · 05cae69f

yt0428 authored Nov 05, 2025


Signed-off-by: yuantao <2422264527@qq.com>
Signed-off-by: yt0428 <51468697+yt0428@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

05cae69f

01 Nov, 2025 1 commit
- [Bugfix] DeepSeek V3.2 MTP metadata & CUDA graph issues (#26779) · d811b442
  Haco authored Nov 01, 2025
```
Signed-off-by: xiaohajiayou <923390377@qq.com>
```
  d811b442
24 Oct, 2025 1 commit

[MISC] `cudagraph_capture_sizes` related improvements (#26016) · 284cc922

fhl2000 authored Oct 24, 2025

Signed-off-by: fhl <2410591650@qq.com>
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

284cc922

23 Oct, 2025 1 commit
- [Chore] Separate out `vllm.utils.platform_utils.py` (#27374) · ca76486a
  Jonathan Chen authored Oct 23, 2025
```
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
```
  ca76486a
18 Oct, 2025 1 commit
- [Misc] Refactor `get_kv_cache_spec` into `AttentionLayerBase` (#26587) · b26b70be
  Nicolò Lucchesi authored Oct 18, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  b26b70be
17 Oct, 2025 1 commit
- [V1][Spec Decode] Fix greedy temperature detection after sampler refactor (#27077) · acedc74b
  Pradyun92 authored Oct 17, 2025
```
Signed-off-by: Pradyun Ramadorai <pradyunr@amazon.com>
Co-authored-by: Pradyun Ramadorai <pradyunr@amazon.com>
```
  acedc74b
15 Oct, 2025 1 commit

[Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change ... · 96b9aa5a

Morrison Turnansky authored Oct 14, 2025


[Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change  compilation level to compilation mode, deprecation compilation level (#26355)
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

96b9aa5a

12 Oct, 2025 1 commit
- Update `Optional[x]` -> `x | None` and `Union[x, y]` to `x | y` (#26633) · 8fcaaf6a
  Harry Mellor authored Oct 12, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8fcaaf6a
10 Oct, 2025 2 commits

[BUG] Qwen3-next MTP. Fix attn metadata build bug (#26564) · e94cfd51
Vadim Gimpelson authored Oct 10, 2025
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
```
e94cfd51

[Spec-Decode] Support piecewise cudagraphs for Eagle head (#25109) · 29255cfc

Lucas Wilkinson authored Oct 10, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>

29255cfc

09 Oct, 2025 2 commits

[Attention][DCP] Support DCP with query length > 1 (MTP) with FA3 (#25049) · 3b736e1c
Ming Yang authored Oct 09, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
3b736e1c

Separate MLAAttention class from Attention (#25103) · e614ab78

Naveenraj Kamalakannan authored Oct 08, 2025

Signed-off-by: Naveenraj Kamalakannan <therealnaveenkamal@gmail.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

e614ab78

06 Oct, 2025 1 commit

[Bugfix][Spec Decode] Fix wrong valid_mask for padded speculation when chunked... · b2ea5ba6

7mile authored Oct 07, 2025


[Bugfix][Spec Decode] Fix wrong valid_mask for padded speculation when chunked prefill occurs (#26231)
Signed-off-by: seven-mile <i@7li.moe>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>

b2ea5ba6

05 Oct, 2025 1 commit
- Convert formatting to use `ruff` instead of `yapf` + `isort` (#26247) · d6953beb
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6953beb
01 Oct, 2025 1 commit
- [Model] MTP fallback to eager for DeepSeek v32 (#25982) · 001e50c9
  Lucia Fang authored Sep 30, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  001e50c9
30 Sep, 2025 1 commit

[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) · fa7e254a

Yongye Zhu authored Sep 30, 2025


Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

fa7e254a

29 Sep, 2025 1 commit
- [XPU]Fix xpu spec decoding UTs, avoid using cuda graph (#25847) · 143844fa
  Kunshang Ji authored Sep 29, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  143844fa
27 Sep, 2025 3 commits

[Bugfix] Merge MM embeddings by index instead of token IDs (#16229) · 27d7638b

Cyrus Leung authored Sep 27, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

27d7638b

[Multimodal][Speculative Decoding]Eagle Eagle3 mm support, enablement on qwen2.5vl (#22872) · f1d53d15

WeiQing Chen authored Sep 27, 2025


Signed-off-by: Junhong <liujunhong11@huawei.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Co-authored-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: LJH-LBJ <98734602+LJH-LBJ@users.noreply.github.com>

f1d53d15

[Spec decode] automatically disable mm for text-only draft models (#25667) · 6f5c0931
Jonas M. Kübler authored Sep 27, 2025
```
Signed-off-by: Jonas Kuebler <kuebj@amazon.com>
```
6f5c0931

26 Sep, 2025 1 commit
- [spec decode] Consolidate speculative decode method name for MTP (#25232) · c70ac4b8
  qizixi authored Sep 26, 2025
```
Signed-off-by: zixi-qi <qizixi@meta.com>
```
  c70ac4b8
25 Sep, 2025 1 commit

[Model] Add LongCat-Flash (#23991) · 845adb3e

XuruiYang authored Sep 25, 2025


Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>

845adb3e

24 Sep, 2025 2 commits

[V0 Deprecation] Remove unused classes in attention (#25541) · e6750d0b

Woosuk Kwon authored Sep 24, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>

e6750d0b

[Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b

Benjamin Chislett authored Sep 23, 2025


Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>

c30b405b

23 Sep, 2025 2 commits

[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) · d5944d51
jiahanc authored Sep 23, 2025
```
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
```
d5944d51

[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed

Lucas Wilkinson authored Sep 23, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

cc1dc7ed