Commits · 93aee29fdb81fa51d5d44f34ea84df80d0764235 · OpenDAS / vllm_cscc

17 Jun, 2025 1 commit
- Fixes IMA for TP w/ flex-attention (#19712) · ddfed314
  Driss Guessous authored Jun 16, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  ddfed314
16 Jun, 2025 2 commits
- [Bugfix] Fix TP inference for Flex attention backend (#19657) · 1173804d
  Isotr0py authored Jun 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1173804d
- [TPU] support attention head dim smaller than 128 (#19620) · a77aea59
  Chengji Yao authored Jun 15, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  a77aea59
15 Jun, 2025 1 commit
- [Kernel] Raise verbose error and consolidate `num_heads/num_kv_heads` divisibility check (#19339) · 0b73736a
  22quinn authored Jun 14, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  0b73736a
13 Jun, 2025 1 commit
- [CUDA] Enable full cudagraph for FlashMLA (#18581) · 3597b06a
  Luka Govedič authored Jun 13, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  3597b06a
12 Jun, 2025 1 commit
- [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) · f98548b9
  Luka Govedič authored Jun 12, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  f98548b9
10 Jun, 2025 1 commit

[BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected... · 467bef18

Rachel Guo authored Jun 10, 2025


[BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected keyword `use_irope` (#19134)
Signed-off-by: Yunqiu Guo <guorachel@meta.com>

467bef18

09 Jun, 2025 1 commit
- [HOT-FIX] Add `kv_sharing_target_layer_name` argument to cutlass_mla backend (#19374) · 8058c911
  Pavani Majety authored Jun 09, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  8058c911
07 Jun, 2025 1 commit
- Add FlexAttention to V1 (#16078) · cf02f9b2
  Driss Guessous authored Jun 07, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  cf02f9b2
05 Jun, 2025 1 commit
- [Misc] Remove unnecessary fallback to prefill-decode attention (#19138) · 18093084
  vllmellm authored Jun 05, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  18093084
04 Jun, 2025 4 commits
- [P/D] Heterogeneous TP (#18833) · b2fac671
  Nicolò Lucchesi authored Jun 05, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
```
  b2fac671
- [Bugfix] Fix FA3 full cuda graph correctness (#19106) · b124e108
  Woosuk Kwon authored Jun 03, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b124e108
- [NVIDIA] Add Cutlass MLA backend (#17625) · 41aa5784
  Kaixi Hou authored Jun 04, 2025
  
  41aa5784
- [CPU] V1 support for the CPU backend (#16441) · 4555143e
  Li, Jiang authored Jun 04, 2025
  
  4555143e
03 Jun, 2025 2 commits
- [V1] Support cross-layer KV sharing (#18212) · bdf13965
  Yong Hoon Shin authored Jun 03, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  bdf13965
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
30 May, 2025 1 commit
- [ROCm] Remove unnecessary assertion of max_model_len in ROCM_AITER_MLA attention backend. (#18938) · 77b6e74f
  vllmellm authored May 30, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  77b6e74f
29 May, 2025 1 commit
- [Attention][V1] Toggle for v1 attention backend (#18275) · da4b69d0
  Gregory Shtrasberg authored May 29, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  da4b69d0
28 May, 2025 1 commit
- [BugFix] FA2 MLA Accuracy Issue (#18807) · ce75efee
  Lucas Wilkinson authored May 28, 2025
```
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>
```
  ce75efee
23 May, 2025 1 commit
- [v1] Redo "Support multiple KV cache groups in GPU model runner (#17945)" (#18593) · 6550114c
  Chen Zhang authored May 24, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  6550114c
21 May, 2025 2 commits
- [FEAT][ROCm] Upgrade AITER MLA v1 backend (#18338) · 94d8ec8d
  vllmellm authored May 22, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  94d8ec8d
- Revert "[v1] Support multiple KV cache groups in GPU model runner (#17945) (#18459) · bb0a3112
  Mark McLoughlin authored May 21, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  bb0a3112
16 May, 2025 1 commit
- [Bugfix][ROCm] Use `chunked_prefill_paged_decode` as fallback for V1 attention on ROCm (#18093) · ee659e3b
  kliuae authored May 16, 2025
```
Signed-off-by: kf <kuanfu.liu@embeddedllm.com>
```
  ee659e3b
15 May, 2025 2 commits
- [Kernel] [V1] Fix performance regression for triton unified attention (#18161) · 01c22335
  Thomas Parnell authored May 15, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  01c22335
- [v1] Support multiple KV cache groups in GPU model runner (#17945) · e60f550b
  Chen Zhang authored May 15, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e60f550b
14 May, 2025 2 commits
- Modularize fused experts and integrate PPLX kernels (#15956) · f9c069c8
  bnellnm authored May 14, 2025
  
  f9c069c8
- [Bugfix][V1] Fix FlashInfer V1 backend using the wrong VllmConfig (#18086) · 12e6c0b4
  Michael Goin authored May 13, 2025
  
  12e6c0b4
11 May, 2025 2 commits
- [BUG] [ROCm] [MLA] Fix variable name bug due to change in variable name in PR #17483 (#17961) · 7de18d54
  TJian authored May 12, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  7de18d54
- [FP8][ROCm][Attention] Enable FP8 KV cache on ROCm for V1 (#17870) · 06c0922a
  Gregory Shtrasberg authored May 11, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  06c0922a
10 May, 2025 1 commit
- [v1] Pass BlockTable and KVCacheSpec to AttentionMetadataBuilders (#17483) · 950751a9
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  950751a9
09 May, 2025 3 commits
- [Bugfix][ROCm] Fix AITER MLA V1 (#17880) · 217db4ba
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  217db4ba
- [Attention] MLA move rotary embedding to cuda-graph region (#17668) · 5e6f9394
  Lucas Wilkinson authored May 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5e6f9394
- [FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) · 3c9396a6
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  3c9396a6
08 May, 2025 2 commits
- [TPU] Fix the test_sampler (#17820) · a463555d
  Jevin Jiang authored May 08, 2025
  
  a463555d
- [Core] Support full cuda graph in v1 (#16072) · 7ea2adb8
  Chanh Nguyen authored May 07, 2025
```
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
```
  7ea2adb8
06 May, 2025 3 commits
- [Kernel] Unified Triton kernel that doesn't distinguish between prefill + decode (#16828) · 2f925e57
  Thomas Parnell authored May 06, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  2f925e57
- [TPU] Increase block size and reset block shapes (#16458) · 621ca2c0
  Jevin Jiang authored May 06, 2025
  
  621ca2c0
- [v1] AttentionMetadata for each layer (#17394) · cba31c47
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cba31c47
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
02 May, 2025 1 commit
- [BugFix][Attention] Fix sliding window attention in V1 giving incorrect results (#17574) · 0f87d8f7
  Lucas Wilkinson authored May 02, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  0f87d8f7