Commits · 96846bb3607370798540c7d325f8d06dbd67dcf4 · OpenDAS / vllm_cscc

09 Jun, 2025 1 commit
- [HOT-FIX] Add `kv_sharing_target_layer_name` argument to cutlass_mla backend (#19374) · 8058c911
  Pavani Majety authored Jun 09, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  8058c911
04 Jun, 2025 1 commit
- [NVIDIA] Add Cutlass MLA backend (#17625) · 41aa5784
  Kaixi Hou authored Jun 04, 2025
  
  41aa5784
03 Jun, 2025 2 commits
- [V1] Support cross-layer KV sharing (#18212) · bdf13965
  Yong Hoon Shin authored Jun 03, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  bdf13965
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
30 May, 2025 1 commit
- [ROCm] Remove unnecessary assertion of max_model_len in ROCM_AITER_MLA attention backend. (#18938) · 77b6e74f
  vllmellm authored May 30, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  77b6e74f
28 May, 2025 1 commit
- [BugFix] FA2 MLA Accuracy Issue (#18807) · ce75efee
  Lucas Wilkinson authored May 28, 2025
```
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>
```
  ce75efee
23 May, 2025 1 commit
- [v1] Redo "Support multiple KV cache groups in GPU model runner (#17945)" (#18593) · 6550114c
  Chen Zhang authored May 24, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  6550114c
21 May, 2025 2 commits
- [FEAT][ROCm] Upgrade AITER MLA v1 backend (#18338) · 94d8ec8d
  vllmellm authored May 22, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  94d8ec8d
- Revert "[v1] Support multiple KV cache groups in GPU model runner (#17945) (#18459) · bb0a3112
  Mark McLoughlin authored May 21, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  bb0a3112
15 May, 2025 1 commit
- [v1] Support multiple KV cache groups in GPU model runner (#17945) · e60f550b
  Chen Zhang authored May 15, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e60f550b
14 May, 2025 1 commit
- Modularize fused experts and integrate PPLX kernels (#15956) · f9c069c8
  bnellnm authored May 14, 2025
  
  f9c069c8
11 May, 2025 1 commit
- [BUG] [ROCm] [MLA] Fix variable name bug due to change in variable name in PR #17483 (#17961) · 7de18d54
  TJian authored May 12, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  7de18d54
10 May, 2025 1 commit
- [v1] Pass BlockTable and KVCacheSpec to AttentionMetadataBuilders (#17483) · 950751a9
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  950751a9
09 May, 2025 3 commits
- [Bugfix][ROCm] Fix AITER MLA V1 (#17880) · 217db4ba
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  217db4ba
- [Attention] MLA move rotary embedding to cuda-graph region (#17668) · 5e6f9394
  Lucas Wilkinson authored May 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5e6f9394
- [FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) · 3c9396a6
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  3c9396a6
06 May, 2025 1 commit
- [v1] AttentionMetadata for each layer (#17394) · cba31c47
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cba31c47
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
02 May, 2025 1 commit
- [Attention] MLA move o_proj q_proj into cuda-graph region (#17484) · afcb3f88
  Lucas Wilkinson authored May 01, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  afcb3f88
29 Apr, 2025 2 commits
- [V1] Remove num_input_tokens from attn_metadata (#17193) · 24e6ad3f
  Chen Zhang authored Apr 30, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  24e6ad3f
- [Bugfix] Add contiguous call inside rope kernel wrapper (#17091) · 17eb306f
  Zhengyuan Su (苏政渊) authored Apr 29, 2025
```
Signed-off-by: 苏政渊 <suzhengyuan@moonshot.cn>
Co-authored-by: 苏政渊 <suzhengyuan@moonshot.cn>
```
  17eb306f
28 Apr, 2025 1 commit

[BugFix] Fix vllm_flash_attn install issues (#17267) · d8bccde6

Lucas Wilkinson authored Apr 27, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>

d8bccde6

22 Apr, 2025 1 commit

[V1] V1 FlashInfer Attention (#16684) · 986537f1

Michael Goin authored Apr 21, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Aurick Qiao <qiao@aurick.net>

986537f1

17 Apr, 2025 2 commits
- [Attention] Update to lastest FA3 code (#13111) · 183dad7a
  Lucas Wilkinson authored Apr 17, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  183dad7a
- [MLA] Simplification to batch P/D reordering (#16673) · 0377b831
  Nick Hill authored Apr 17, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  0377b831
11 Apr, 2025 1 commit
- [Kernel] support merge_attn_states CUDA kernel, 3x speedup (#16173) · e9528f6d
  DefTruth authored Apr 11, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  e9528f6d
09 Apr, 2025 1 commit
- [BugFix] fix some typos found by typos. (#16314) · 04149cce
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  04149cce
23 Mar, 2025 1 commit

[V1] Enable V1 Fp8 cache for FA3 in the oracle (#15191) · dccf535f

Lucas Wilkinson authored Mar 23, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

dccf535f

21 Mar, 2025 2 commits
- [V1] Fix wrong import path of get_flash_attn_version (#15280) · 91ca929d
  Lehua Ding authored Mar 21, 2025
```
Signed-off-by: Lehua Ding <lehuading@tencent.com>
```
  91ca929d
- [V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250) · 0c6f5023
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  0c6f5023
14 Mar, 2025 1 commit
- [Attention] MLA get rid of materialization (#14770) · 9532c498
  Lucas Wilkinson authored Mar 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9532c498
11 Mar, 2025 1 commit
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
10 Mar, 2025 1 commit
- [Perf] Improve MLA on V1 (#14540) · fb0acb6c
  Simon Mo authored Mar 10, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  fb0acb6c
08 Mar, 2025 2 commits
- [Bugfix] DeepSeek Accuracy (#14476) · db84f5eb
  Lucas Wilkinson authored Mar 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  db84f5eb
- Revert "[Perf] Reduce MLA CPU overheads in V1 (#14384)" (#14471) · ca7a2d5f
  Tyler Michael Smith authored Mar 08, 2025
  
  ca7a2d5f
07 Mar, 2025 2 commits
- [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) · e1744502
  Luka Govedič authored Mar 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  e1744502
- [Perf] Reduce MLA CPU overheads in V1 (#14384) · dae68969
  Lucas Wilkinson authored Mar 06, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dae68969
06 Mar, 2025 3 commits
- [V1][Bugfix] Standardize quantized kv cache rejection for attention backends (#14221) · 6832707e
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6832707e
- Fix mla prefill context performance (#13897) · 9f1710f1
  Ying Zhong authored Mar 07, 2025
```
Signed-off-by: ZhongYingMatrix <zhongyingmatrix@gmail.com>
```
  9f1710f1
- [BugFix] MLA + V1, illegal memory access and accuracy issues (#14253) · f6bb18fd
  Lucas Wilkinson authored Mar 05, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  f6bb18fd