Commits · 2e41f5abca796c615f4db8d9a496d037fa385653 · OpenDAS / vllm_cscc

15 Sep, 2025 1 commit
- [XPU] Set consistent default KV cache layout (#24745) · 2e41f5ab
  Nicolò Lucchesi authored Sep 15, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  2e41f5ab
12 Sep, 2025 3 commits
- [Qwen3Next] Fixes the cuda graph capture conditions under large batch sizes (#24660) (#24667) · 8226dd56
  Tao He authored Sep 13, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
```
  8226dd56
- [Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705) · 7ba32aa6
  Matthew Bonanni authored Sep 12, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
```
  7ba32aa6
- [Kernel] [CPU] refactor `cpu_attn.py:_run_sdpa_forward` for better memory access (#24701) · 7a1c4025
  Ignacio Sica authored Sep 12, 2025
```
Signed-off-by: ignaciosica <mignacio.sica@gmail.com>
```
  7a1c4025
11 Sep, 2025 4 commits
- [flashinfer] [kernel] support for fp8 kv cache for trtllm prefill attention (#24197) · e42af78b
  Xiaozhu Meng authored Sep 11, 2025
```
Signed-off-by: Xiaozhu <mxz297@gmail.com>
```
  e42af78b
- fix some typos (#24616) · e26fef83
  co63oc authored Sep 12, 2025
```
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
```
  e26fef83
- Add the support for the qwen3 next model (a hybrid attention model). (#24526) · e93f4cc9
  Tao He authored Sep 11, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  e93f4cc9
- [Doc]: fixing doc typos (#24635) · e2b1f863
  Didier Durand authored Sep 11, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  e2b1f863
10 Sep, 2025 8 commits

[Kernel] Flashinfer MLA (trtllm-gen) decode kernel integration (#21078) · dcb28a33

Hanjie Qiu authored Sep 10, 2025


Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

dcb28a33

[Perf] Warmup FlashInfer attention during startup (#23439) · fba78565

Michael Goin authored Sep 10, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>

fba78565

[gpt-oss] raise error for flashinfer backend without trtllm (#24482) · b5e383cd
Chen Zhang authored Sep 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
b5e383cd

[torch.compile][ROCm][V1] Enable attention output FP8 fusion for V1 attention backends (#19767) · 9a161307

Gregory Shtrasberg authored Sep 10, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

9a161307

[v1] Add Whisper model support (encoder-decoder) (#21088) · 37e8182b
Russell Bryant authored Sep 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
```
37e8182b
[Bugfix] Enable FP8 KV cache for FlashInfer and Triton backend on non-sm100 GPUs (#24577) · a0933c3b
Thien Tran authored Sep 11, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
a0933c3b

[Attention] add DCP support for FLASH_ATTN_MLA backend (#24453) · 0ae43dbf

Lucas Wilkinson authored Sep 10, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>

0ae43dbf

[Perf] Convert np array to torch tensor to index into block table for attn chunking (#24474) · dc625ea6
Yong Hoon Shin authored Sep 09, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
dc625ea6

09 Sep, 2025 2 commits

[Feature] Disallow FlashMLA on Blackwell (#24521) · 15de5ff9

Wentao Ye authored Sep 09, 2025

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

15de5ff9

[Flashinfer] Support Flashinfer TRTLLM FP8-qkv BF16/FP16-out Attention Kernel (#23647) · bba1042c
elvischenv authored Sep 09, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
bba1042c

08 Sep, 2025 3 commits

[Attention] FlashAttention MLA cudagraph support (#23958) · 620db1fc

Matthew Bonanni authored Sep 08, 2025


Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

620db1fc

[Bugfix] Fix mamba2 prefill chunking (#23279) · e0413141

tomeras91 authored Sep 08, 2025

Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

e0413141

[Kernel] Support decode context parallelism on Blackwell with CUTLASS MLA (#24385) · 86173ad5

Ming Yang authored Sep 07, 2025


Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

86173ad5

07 Sep, 2025 1 commit
- [attention][DCP] use AttentionImpl.need_to_return_lse_for_decode (#24372) · 558f0907
  youkaichao authored Sep 07, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  558f0907
06 Sep, 2025 2 commits

[Feature] Support Decode Context Parallel (DCP) for MLA (#23734) · ac201a0e

yzds authored Sep 06, 2025


Signed-off-by: hongchao <hongchao@msh.team>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: hongchao <hongchao@msh.team>
Co-authored-by: youkaichao <youkaichao@gmail.com>

ac201a0e

[Doc]: fix typos in Python comments (#24294) · 35bf1938

Didier Durand authored Sep 06, 2025


Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

35bf1938

04 Sep, 2025 3 commits

[Doc]: fix typos in Python comments (#24173) · 83609ca9

Didier Durand authored Sep 04, 2025


Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

83609ca9

[XPU] support Triton Attention backend on Intel GPU (#24149) · 16ded21e
Kunshang Ji authored Sep 04, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
16ded21e

[Attention] FlashAttn MLA (#14258) · 402759d4

Lucas Wilkinson authored Sep 04, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>

402759d4

03 Sep, 2025 3 commits
- [Attention] Blackwell FP8 MLA support with CUTLASS_MLA backend (#23289) · a7423220
  Matthew Bonanni authored Sep 03, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  a7423220
- [Doc]: fix typos in Python comments (#24093) · d7e1e599
  Didier Durand authored Sep 03, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  d7e1e599
- fix some typos (#24071) · 1bd007f2
  co63oc authored Sep 03, 2025
```
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
```
  1bd007f2
30 Aug, 2025 1 commit
- [Misc] add reorder_batch AttentionMetadataBuilder (#23798) · fb4983e1
  Ning Xie authored Aug 30, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  fb4983e1
29 Aug, 2025 1 commit

Update PyTorch to 2.8.0 (#20358) · 67c14906

Huy Do authored Aug 29, 2025


Signed-off-by: Huy Do <huydhn@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

67c14906

28 Aug, 2025 3 commits
- [V1] Enable prefill optimization for Gemma3n (#22628) · cb293f6a
  Yong Hoon Shin authored Aug 28, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  cb293f6a
- [BugFix][FlashInfer] Fix potential race condition for paged_kv_indptr_cpu (#23737) · 7ffbf272
  Woosuk Kwon authored Aug 28, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7ffbf272
- [ROCm][Aiter] Add triton fp8 bmm kernel for mla (#23264) · 04d1dd7f
  Divakar Verma authored Aug 28, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: ShaoChunLee <Shao-Chun.Lee@amd.com>
```
  04d1dd7f
27 Aug, 2025 3 commits
- [Docs] Fix warnings in `mkdocs build` (continued) (#23743) · 4e4d017b
  Hyogeun Oh (오효근) authored Aug 28, 2025
```
Signed-off-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>
```
  4e4d017b
- [FlashInfer] Cache hyper params in metadata builder (#23732) · 11eddf02
  Woosuk Kwon authored Aug 27, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  11eddf02
- Optimize input preparation for FlashInfer [2/N] (#23174) · 6578e873
  Woosuk Kwon authored Aug 27, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6578e873
25 Aug, 2025 2 commits
- [Misc] Simplify FlashInfer attention metadata (#23585) · efc88cf6
  Woosuk Kwon authored Aug 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  efc88cf6
- Updates to Flex + VLLm integration (#21416) · e0329ed4
  Driss Guessous authored Aug 25, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  e0329ed4