Commits · 4e4d017b6f70c729e7c78f74e4328a4ebca7b8ec · OpenDAS / vllm_cscc

27 Aug, 2025 3 commits
- [Docs] Fix warnings in `mkdocs build` (continued) (#23743) · 4e4d017b
  Hyogeun Oh (오효근) authored Aug 28, 2025
```
Signed-off-by: Zerohertz <ohg3417@gmail.com>
Signed-off-by: Hyogeun Oh (오효근) <ohg3417@gmail.com>
```
  4e4d017b
- [FlashInfer] Cache hyper params in metadata builder (#23732) · 11eddf02
  Woosuk Kwon authored Aug 27, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  11eddf02
- Optimize input preparation for FlashInfer [2/N] (#23174) · 6578e873
  Woosuk Kwon authored Aug 27, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6578e873
25 Aug, 2025 1 commit
- [Misc] Simplify FlashInfer attention metadata (#23585) · efc88cf6
  Woosuk Kwon authored Aug 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  efc88cf6
22 Aug, 2025 1 commit

[NVIDIA][torch.compile] Support Flashinfer TRTLLM FP8-q/kv NVFP4-out Attention Kernel (#22703) · 24d0c9e6

elvischenv authored Aug 23, 2025


Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

24d0c9e6

21 Aug, 2025 1 commit
- [Core] Always use tensor cores for Flashinfer Decode Wrapper (#23214) · 1d353b63
  Pavani Majety authored Aug 21, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  1d353b63
20 Aug, 2025 1 commit
- [Misc] Add max_seq_len to CommonAttentionMetadata (#23216) · d6d13bd4
  Woosuk Kwon authored Aug 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d6d13bd4
19 Aug, 2025 3 commits
- [Misc] Minor refactoring for FlashInfer backend (#23147) · e61bac87
  Woosuk Kwon authored Aug 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e61bac87
- [Misc] Enable yapf for FlashInfer backend (#23193) · 5b5f350d
  Woosuk Kwon authored Aug 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  5b5f350d
- [NVIDIA] Support Flashinfer TRTLLM FP8-q/kv/out Attention Kernel (#21716) · 03752dba
  elvischenv authored Aug 19, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  03752dba
16 Aug, 2025 1 commit
- [Bugfix gpt-oss] Fix float32 convert for flashinfer sink support (#23016) · 000cceca
  Michael Goin authored Aug 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  000cceca
15 Aug, 2025 2 commits

minor: zero workspace buffer init for flashinfer trtllm-gen attn (#22603) · 1723ef1a
eigen authored Aug 15, 2025

1723ef1a

[Core] Allow full cudagraph with separate attention routines and orthogonal to... · 74f441f4

fhl2000 authored Aug 15, 2025


[Core] Allow full cudagraph with separate attention routines and orthogonal to compilation, add support for FA2 and FlashInfer (#20059)
Signed-off-by: fhl <2410591650@qq.com>
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>

74f441f4

13 Aug, 2025 1 commit
- Force TRTLLM attention for gpt-oss on SM100 (#22678) · c6b92879
  Michael Goin authored Aug 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  c6b92879
08 Aug, 2025 1 commit
- [bugfix] Fix Llama3/4 issues caused by FlashInfer 0.2.10 (#22426) · af473f0a
  Po-Han Huang (NVIDIA) authored Aug 08, 2025
```
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
```
  af473f0a
07 Aug, 2025 1 commit

Fix trtllm-gen attention env and add attention sink (#22378) · 9a3835aa

Lain authored Aug 06, 2025


Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: Lain <fusiyuan2000@hotmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

9a3835aa

06 Aug, 2025 2 commits

[gpt-oss] Enhance error msg on attention sink init (#22335) · 31f5dc5b

Yongye Zhu authored Aug 06, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

31f5dc5b

[gpt-oss] flashinfer attention sink init (#22330) · 90ec0069

Yongye Zhu authored Aug 05, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>

90ec0069

05 Aug, 2025 1 commit
- [NVIDIA] Support Flashinfer TRT-LLM Prefill Attention Kernel (#22095) · 83156c7b
  elvischenv authored Aug 05, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  83156c7b
02 Aug, 2025 2 commits
- [V1] [Hybrid] Validate compatibility of attention backend batch reordering at init time (#21557) · 4abfd879
  Thomas Parnell authored Aug 02, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  4abfd879
- [V1][CUDA] Full cudagraph support for FlashInfer (#21367) · 23322431
  fhl2000 authored Aug 02, 2025
  
  23322431
01 Aug, 2025 1 commit
- [Bugfix] Check NVIDIA artifactory is accessible before using flashinfer cubin kernels (#21893) · f81c1bb0
  Michael Goin authored Aug 01, 2025
  
  f81c1bb0
30 Jul, 2025 1 commit
- [v1][attention] Support Hybrid Allocator + FlashInfer (#21412) · 555e7225
  Chen Zhang authored Jul 29, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  555e7225
29 Jul, 2025 1 commit
- [Bugfix] Fix workspace buffer None issue for Flashinfer TRTLLM Backend (#21525) · 58b11b24
  elvischenv authored Jul 29, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  58b11b24
24 Jul, 2025 2 commits
- update flashinfer to v0.2.9rc1 (#21485) · 2dd72d23
  weiliang authored Jul 25, 2025
```
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
```
  2dd72d23
- [Attention] Optimize FlashInfer MetadataBuilder Build call (#21137) · 61b8cea3
  Lucas Wilkinson authored Jul 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  61b8cea3
21 Jul, 2025 1 commit

[Attention] Clean up iRoPE in V1 (#21188) · 304dce7e

Lucas Wilkinson authored Jul 21, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

304dce7e

19 Jul, 2025 1 commit
- [V0 Deprecation] Deprecate BlockSparse Attention & Phi3-Small (#21217) · 752c6ade
  Woosuk Kwon authored Jul 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  752c6ade
18 Jul, 2025 2 commits
- [Attention] Make local attention backend agnostic (#21093) · 89cab4d0
  Lucas Wilkinson authored Jul 18, 2025
  
  89cab4d0
- [Bugfix] Fix the tensor non-contiguous issue for Flashinfer TRT-LLM backend... · 8dfb45ca
  elvischenv authored Jul 18, 2025
```
[Bugfix] Fix the tensor non-contiguous issue for Flashinfer TRT-LLM backend attention kernel (#21133)
```
  8dfb45ca
17 Jul, 2025 1 commit
- [Attention] Refactor attention metadata builder interface (#20466) · 76b49444
  Lucas Wilkinson authored Jul 17, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  76b49444
14 Jul, 2025 1 commit
- [Misc] Log the reason for falling back to FlexAttention (#20699) · e8cc53af
  Cyrus Leung authored Jul 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e8cc53af
11 Jul, 2025 2 commits
- [Core] Add Flashinfer TRTLLM Backend for Flashinfer decode path (SM100). (#19825) · 7bd4c37a
  Pavani Majety authored Jul 11, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: shuw <shuw@nvidia.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  7bd4c37a
- [Attention] MLA - Flashinfer Ragged Prefill (#20034) · 5b032352
  Alexander Matveev authored Jul 10, 2025
  
  5b032352
06 Jul, 2025 1 commit
- [V1] Support any head size for FlexAttention backend (#20467) · 9fb52e52
  Cyrus Leung authored Jul 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9fb52e52
17 Jun, 2025 1 commit
- [V1][Kernel] Flashinfer HND KV cache layout (#19280) · 4c8f64fa
  Nicolò Lucchesi authored Jun 17, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  4c8f64fa
15 Jun, 2025 1 commit
- [Kernel] Raise verbose error and consolidate `num_heads/num_kv_heads` divisibility check (#19339) · 0b73736a
  22quinn authored Jun 14, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  0b73736a
13 Jun, 2025 1 commit
- [CUDA] Enable full cudagraph for FlashMLA (#18581) · 3597b06a
  Luka Govedič authored Jun 13, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  3597b06a
12 Jun, 2025 1 commit
- [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) · f98548b9
  Luka Govedič authored Jun 12, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  f98548b9
10 Jun, 2025 1 commit

[BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected... · 467bef18

Rachel Guo authored Jun 10, 2025


[BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected keyword `use_irope` (#19134)
Signed-off-by: Yunqiu Guo <guorachel@meta.com>

467bef18