Commits · 66e674cdd549e89b341fb33405a2c02ec5f5ae8d · OpenDAS / vllm_cscc

05 Dec, 2025 2 commits

[Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments (#26315) · 66e674cd

Matthew Bonanni authored Dec 05, 2025

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>

66e674cd

[Bugfix] Correct num_q_heads on DCP for Flashinfer backends (#29487) · d698bb38

Jingchun Gao authored Dec 05, 2025


Signed-off-by: Jingchun Gao <gaojingchun1@huawei.com>
Signed-off-by: Jingchun Gao <63247409+gjc0824@users.noreply.github.com>
Co-authored-by: Jingchun Gao <gaojingchun1@huawei.com>

d698bb38

28 Nov, 2025 1 commit
- bugfix: correct attn output with base 2 or e (#28840) · 9726e645
  Augusto Yao authored Nov 29, 2025
```
Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>
```
  9726e645
27 Nov, 2025 1 commit

[BugFix] Fix `plan` API Mismatch when using latest FlashInfer (#29426) · a5345bf4

Andrii Skliar authored Nov 27, 2025


Signed-off-by: Andrii Skliar <askliar@askliar-mlt.client.nvidia.com>
Co-authored-by: Andrii Skliar <askliar@askliar-mlt.client.nvidia.com>

a5345bf4

26 Nov, 2025 1 commit
- [Core] Refactor padding logic and pad for CUDA graphs before attention metadata building (#28579) · 56539cdd
  Lucas Wilkinson authored Nov 26, 2025
  
  56539cdd
22 Nov, 2025 1 commit
- [Attention] Refactor FA `block_size` limitations to hybrid models only (#29084) · 066209a0
  Nicolò Lucchesi authored Nov 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  066209a0
21 Nov, 2025 1 commit
- GPU Model Runner V2 (#25266) · 30b44a15
  Woosuk Kwon authored Nov 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  30b44a15
20 Nov, 2025 1 commit
- [KVConnector][Core] Support cross-layer KV blocks (#27743) · 64746471
  Or Ozeri authored Nov 20, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  64746471
14 Nov, 2025 1 commit

[DCP] Support Decode Context Parallel (DCP) for GQA with Flashinfer (#25438) · 4516d44b

Jingchun Gao authored Nov 14, 2025


Signed-off-by: gaojc <1055866782@qq.com>
Signed-off-by: Jingchun Gao <gaojingchun1@huawei.com>
Signed-off-by: Jingchun Gao <63247409+gjc0824@users.noreply.github.com>
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: gaojingchun (A) <g00955623@china.huawei.com>
Co-authored-by: Jingchun Gao <gaojingchun1@huawei.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>

4516d44b

13 Nov, 2025 1 commit
- [Bugfix] Fix SM100 gpt-oss regression due to faulty attn sink support (#28561) · a543e678
  Michael Goin authored Nov 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a543e678
12 Nov, 2025 1 commit

[Perf] Refactor cudagraph_support to enable full CUDA graphs for spec decoding... · 30441957

Benjamin Chislett authored Nov 12, 2025


[Perf] Refactor cudagraph_support to enable full CUDA graphs for spec decoding with FlashInfer (#28479)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>

30441957

11 Nov, 2025 2 commits

[Feature] Allow configuring FlashInfer workspace size (#28269) · 412e153d

Max Hu authored Nov 11, 2025


Signed-off-by: Max Hu <hyoung2991@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

412e153d

[Attention] Refactor CUDA attention backend selection logic (#24794) · b30dfa03

Matthew Bonanni authored Nov 11, 2025


Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

b30dfa03

10 Nov, 2025 1 commit
- [V0 deprecation] Remove no longer used `get_metadata_cls` (#28370) · e8697faf
  Lucas Wilkinson authored Nov 10, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  e8697faf
05 Nov, 2025 1 commit
- [FlashInfer] Avoid FlashInfer block_size 16 + head_size 256 on blackwell (#27994) · c765f0b4
  Chen Zhang authored Nov 05, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  c765f0b4
26 Oct, 2025 1 commit

[Chore]:Extract math and argparse utilities to separate modules (#27188) · 71b1c8b6

Yeshwanth N authored Oct 26, 2025


Signed-off-by: Yeshwanth Surya <yeshsurya@gmail.com>
Signed-off-by: Yeshwanth N <yeshsurya@gmail.com>
Signed-off-by: yeshsurya <yeshsurya@gmail.com>

71b1c8b6

24 Oct, 2025 1 commit

[MISC] `cudagraph_capture_sizes` related improvements (#26016) · 284cc922

fhl2000 authored Oct 24, 2025

Signed-off-by: fhl <2410591650@qq.com>
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

284cc922

23 Oct, 2025 1 commit
- [Chore] Separate out `vllm.utils.platform_utils.py` (#27374) · ca76486a
  Jonathan Chen authored Oct 23, 2025
```
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
```
  ca76486a
16 Oct, 2025 2 commits
- [small][batch invariance] Rename the env and internal flags to simplify usage (#26855) · b2f78cba
  Bram Wasti authored Oct 16, 2025
```
Signed-off-by: Bram Wasti <bwasti@meta.com>
```
  b2f78cba
- [Feature] Add process_weights_after_loading to AttentionImpl (#26870) · 5afd3276
  rongfu.leng authored Oct 16, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  5afd3276
15 Oct, 2025 1 commit

Move query quantization to attention layer for Flashinfer & Triton. (#26534) · 0a9ef0cf

Adrian Abeyta authored Oct 15, 2025


Signed-off-by: adabeyta <aabeyta@redhat.com>
Signed-off-by: Adrian Abeyta <aabeyta@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

0a9ef0cf

14 Oct, 2025 1 commit

remove attn output view kernel (#26680) · a86b4c58

Boyuan Feng authored Oct 14, 2025

Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

a86b4c58

13 Oct, 2025 1 commit
- [unrevert] Add batch invariant kernel override for FlashInfer backend [2/n] (#26373) · 32637990
  Bram Wasti authored Oct 13, 2025
```
Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
```
  32637990
12 Oct, 2025 1 commit
- Update `Optional[x]` -> `x | None` and `Union[x, y]` to `x | y` (#26633) · 8fcaaf6a
  Harry Mellor authored Oct 12, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8fcaaf6a
09 Oct, 2025 3 commits
- [Bugfix] Fix CUDA graph selection bug in FlashInfer at high concurrency (#26499) · 6e783bc5
  Benjamin Chislett authored Oct 09, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  6e783bc5
- Bump Flashinfer to v0.4.0 (#26326) · 5e49c3e7
  elvischenv authored Oct 09, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  5e49c3e7
- [Hybrid]: Decouple Kernel Block Size from KV Page Size (#24486) · d24cf322
  Zhiyuan Li authored Oct 09, 2025
```
Signed-off-by: lizhiyuan <uniartisan2017@gmail.com>
Signed-off-by: Zhiyuan Li <uniartisan2017@gmail.com>
```
  d24cf322
08 Oct, 2025 1 commit

[Bugfix][Flashinfer] fix VLLM_USE_TRTLLM_ATTENTION issue for models with diff... · b82f4307

elvischenv authored Oct 09, 2025


[Bugfix][Flashinfer] fix VLLM_USE_TRTLLM_ATTENTION issue for models with diff hyperparameters (#25924)
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>

b82f4307

06 Oct, 2025 1 commit
- [Attention] Remove unused reorder_batch method (#24463) · 4727a8af
  Matthew Bonanni authored Oct 06, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  4727a8af
05 Oct, 2025 3 commits
- Fix per file ruff ignores related to typing (#26254) · 1c0c6820
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  1c0c6820
- Remove all references to `yapf` as it's no longer used (#26251) · 4e256cad
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4e256cad
- Convert formatting to use `ruff` instead of `yapf` + `isort` (#26247) · d6953beb
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6953beb
04 Oct, 2025 2 commits
- Revert "Add batch invariant kernel override for FlashInfer backend [2/n]" (#26220) · 1838cd48
  Cyrus Leung authored Oct 04, 2025
  
  1838cd48
- Add batch invariant kernel override for FlashInfer backend [2/n] (#25769) · 2f7dbc9b
  Bram Wasti authored Oct 03, 2025
```
Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  2f7dbc9b
02 Oct, 2025 1 commit

[Bugfix] Disable cascade attention with FlashInfer (#26130) · f1fc2107

Michael Goin authored Oct 02, 2025

Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

f1fc2107

30 Sep, 2025 1 commit

[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) · fa7e254a

Yongye Zhu authored Sep 30, 2025


Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

fa7e254a

25 Sep, 2025 1 commit
- [V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) · 3468f17e
  Matthew Bonanni authored Sep 25, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
```
  3468f17e
24 Sep, 2025 2 commits
- [Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b
  Benjamin Chislett authored Sep 23, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
```
  c30b405b
- [Bugfix] Use a separate FlashInfer workspace buffer for trtllm-gen (#25520) · 19836092
  Benjamin Chislett authored Sep 23, 2025
  
  19836092
19 Sep, 2025 1 commit
- [BugFix] Make FlashInferMetadataBuilder non-blocking (#25040) · b1a63d1b
  nvjullin authored Sep 20, 2025
```
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  b1a63d1b