Commits · 2f7dbc9b42c51ba192e3dded515e4e07cdfdabea · OpenDAS / vllm_cscc

04 Oct, 2025 1 commit

Add batch invariant kernel override for FlashInfer backend [2/n] (#25769) · 2f7dbc9b

Bram Wasti authored Oct 03, 2025


Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

2f7dbc9b

02 Oct, 2025 1 commit

[Bugfix] Disable cascade attention with FlashInfer (#26130) · f1fc2107

Michael Goin authored Oct 02, 2025

Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

f1fc2107

30 Sep, 2025 1 commit

[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) · fa7e254a

Yongye Zhu authored Sep 30, 2025


Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

fa7e254a

25 Sep, 2025 1 commit
- [V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) · 3468f17e
  Matthew Bonanni authored Sep 25, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
```
  3468f17e
24 Sep, 2025 2 commits
- [Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b
  Benjamin Chislett authored Sep 23, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
```
  c30b405b
- [Bugfix] Use a separate FlashInfer workspace buffer for trtllm-gen (#25520) · 19836092
  Benjamin Chislett authored Sep 23, 2025
  
  19836092
19 Sep, 2025 1 commit
- [BugFix] Make FlashInferMetadataBuilder non-blocking (#25040) · b1a63d1b
  nvjullin authored Sep 20, 2025
```
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  b1a63d1b
17 Sep, 2025 1 commit

[Bugfix] Refactor Flashinfer TRTLLM attention kernel selection logic (#24600) · e67a79db

elvischenv authored Sep 18, 2025


Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

e67a79db

11 Sep, 2025 1 commit
- [flashinfer] [kernel] support for fp8 kv cache for trtllm prefill attention (#24197) · e42af78b
  Xiaozhu Meng authored Sep 11, 2025
```
Signed-off-by: Xiaozhu <mxz297@gmail.com>
```
  e42af78b
10 Sep, 2025 4 commits

[Perf] Warmup FlashInfer attention during startup (#23439) · fba78565

Michael Goin authored Sep 10, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>

fba78565

[gpt-oss] raise error for flashinfer backend without trtllm (#24482) · b5e383cd
Chen Zhang authored Sep 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
b5e383cd
[v1] Add Whisper model support (encoder-decoder) (#21088) · 37e8182b
Russell Bryant authored Sep 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
```
37e8182b
[Bugfix] Enable FP8 KV cache for FlashInfer and Triton backend on non-sm100 GPUs (#24577) · a0933c3b
Thien Tran authored Sep 11, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
a0933c3b

09 Sep, 2025 1 commit
- [Flashinfer] Support Flashinfer TRTLLM FP8-qkv BF16/FP16-out Attention Kernel (#23647) · bba1042c
  elvischenv authored Sep 09, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  bba1042c
06 Sep, 2025 1 commit

[Doc]: fix typos in Python comments (#24294) · 35bf1938

Didier Durand authored Sep 06, 2025


Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

35bf1938

04 Sep, 2025 1 commit

[Attention] FlashAttn MLA (#14258) · 402759d4

Lucas Wilkinson authored Sep 04, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>

402759d4

03 Sep, 2025 1 commit
- fix some typos (#24071) · 1bd007f2
  co63oc authored Sep 03, 2025
```
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
```
  1bd007f2
28 Aug, 2025 1 commit
- [BugFix][FlashInfer] Fix potential race condition for paged_kv_indptr_cpu (#23737) · 7ffbf272
  Woosuk Kwon authored Aug 28, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7ffbf272
27 Aug, 2025 3 commits
- [Docs] Fix warnings in `mkdocs build` (continued) (#23743) · 4e4d017b
  Hyogeun Oh (오효근) authored Aug 28, 2025
```
Signed-off-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>
```
  4e4d017b
- [FlashInfer] Cache hyper params in metadata builder (#23732) · 11eddf02
  Woosuk Kwon authored Aug 27, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  11eddf02
- Optimize input preparation for FlashInfer [2/N] (#23174) · 6578e873
  Woosuk Kwon authored Aug 27, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6578e873
25 Aug, 2025 1 commit
- [Misc] Simplify FlashInfer attention metadata (#23585) · efc88cf6
  Woosuk Kwon authored Aug 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  efc88cf6
22 Aug, 2025 1 commit

[NVIDIA][torch.compile] Support Flashinfer TRTLLM FP8-q/kv NVFP4-out Attention Kernel (#22703) · 24d0c9e6

elvischenv authored Aug 23, 2025


Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

24d0c9e6

21 Aug, 2025 1 commit
- [Core] Always use tensor cores for Flashinfer Decode Wrapper (#23214) · 1d353b63
  Pavani Majety authored Aug 21, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  1d353b63
20 Aug, 2025 1 commit
- [Misc] Add max_seq_len to CommonAttentionMetadata (#23216) · d6d13bd4
  Woosuk Kwon authored Aug 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d6d13bd4
19 Aug, 2025 3 commits
- [Misc] Minor refactoring for FlashInfer backend (#23147) · e61bac87
  Woosuk Kwon authored Aug 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e61bac87
- [Misc] Enable yapf for FlashInfer backend (#23193) · 5b5f350d
  Woosuk Kwon authored Aug 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  5b5f350d
- [NVIDIA] Support Flashinfer TRTLLM FP8-q/kv/out Attention Kernel (#21716) · 03752dba
  elvischenv authored Aug 19, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  03752dba
16 Aug, 2025 1 commit
- [Bugfix gpt-oss] Fix float32 convert for flashinfer sink support (#23016) · 000cceca
  Michael Goin authored Aug 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  000cceca
15 Aug, 2025 2 commits

minor: zero workspace buffer init for flashinfer trtllm-gen attn (#22603) · 1723ef1a
eigen authored Aug 15, 2025

1723ef1a

[Core] Allow full cudagraph with separate attention routines and orthogonal to... · 74f441f4

fhl2000 authored Aug 15, 2025


[Core] Allow full cudagraph with separate attention routines and orthogonal to compilation, add support for FA2 and FlashInfer (#20059)
Signed-off-by: fhl <2410591650@qq.com>
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>

74f441f4

13 Aug, 2025 1 commit
- Force TRTLLM attention for gpt-oss on SM100 (#22678) · c6b92879
  Michael Goin authored Aug 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  c6b92879
08 Aug, 2025 1 commit
- [bugfix] Fix Llama3/4 issues caused by FlashInfer 0.2.10 (#22426) · af473f0a
  Po-Han Huang (NVIDIA) authored Aug 08, 2025
```
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
```
  af473f0a
07 Aug, 2025 1 commit

Fix trtllm-gen attention env and add attention sink (#22378) · 9a3835aa

Lain authored Aug 06, 2025


Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: Lain <fusiyuan2000@hotmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

9a3835aa

06 Aug, 2025 2 commits

[gpt-oss] Enhance error msg on attention sink init (#22335) · 31f5dc5b

Yongye Zhu authored Aug 06, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

31f5dc5b

[gpt-oss] flashinfer attention sink init (#22330) · 90ec0069

Yongye Zhu authored Aug 05, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>

90ec0069

05 Aug, 2025 1 commit
- [NVIDIA] Support Flashinfer TRT-LLM Prefill Attention Kernel (#22095) · 83156c7b
  elvischenv authored Aug 05, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  83156c7b
02 Aug, 2025 2 commits
- [V1] [Hybrid] Validate compatibility of attention backend batch reordering at init time (#21557) · 4abfd879
  Thomas Parnell authored Aug 02, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  4abfd879
- [V1][CUDA] Full cudagraph support for FlashInfer (#21367) · 23322431
  fhl2000 authored Aug 02, 2025
  
  23322431
01 Aug, 2025 1 commit
- [Bugfix] Check NVIDIA artifactory is accessible before using flashinfer cubin kernels (#21893) · f81c1bb0
  Michael Goin authored Aug 01, 2025
  
  f81c1bb0