Commits · eea22a56ab08fb018e8fc51d1bf988cb85f37115 · OpenDAS / vllm_cscc

11 May, 2025 1 commit
- fix amd triton mla path (#17871) · eea22a56
  Shiyan Deng authored May 11, 2025
  
  eea22a56
09 May, 2025 3 commits
- Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910) · 85b72cb7
  Michael Goin authored May 09, 2025
  
  85b72cb7
- [BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864) · 9f64e934
  qli88 authored May 09, 2025
```
Signed-off-by: Qiang Li <qiang.li2@amd.com>
```
  9f64e934
- [Attention] MLA move rotary embedding to cuda-graph region (#17668) · 5e6f9394
  Lucas Wilkinson authored May 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5e6f9394
08 May, 2025 1 commit
- [Hardware][Intel-Gaudi] Support Automatic Prefix Caching on HPU (#17648) · 843b2227
  Agata Dobrzyniewicz authored May 08, 2025
```
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
```
  843b2227
07 May, 2025 1 commit
- [TPU] Add kernel test for moe_pallas (#17496) · e50a1f1a
  Michael Goin authored May 06, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  e50a1f1a
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
02 May, 2025 2 commits

[Core] [Bugfix] Add Input Embeddings (#15428) · cc2a77d7

Andrew Sansom authored May 02, 2025


Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: 临景 <linjing.yx@alibaba-inc.com>
Co-authored-by: Bryce1010 <bryceyx@gmail.com>
Co-authored-by: Nan2018 <nan@protopia.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

cc2a77d7

[Attention] MLA move o_proj q_proj into cuda-graph region (#17484) · afcb3f88
Lucas Wilkinson authored May 01, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
afcb3f88

01 May, 2025 1 commit
- [BugFix] Fix mla cpu - missing 3 required positional arguments (#17494) · 3c3d7672
  Lucas Wilkinson authored May 01, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  3c3d7672
30 Apr, 2025 1 commit

[Hardware][Intel GPU] Upgrade to torch 2.7 (#17444) · ed6cfb90

Kunshang Ji authored Apr 30, 2025


Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Qiming Zhang <qiming1.zhang@intel.com>

ed6cfb90

28 Apr, 2025 1 commit

[BugFix] Fix vllm_flash_attn install issues (#17267) · d8bccde6

Lucas Wilkinson authored Apr 27, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>

d8bccde6

27 Apr, 2025 1 commit
- [Bugfix] Get a specific type of layer from forward context (#17222) · 838cedad
  Chen Zhang authored Apr 27, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  838cedad
26 Apr, 2025 2 commits
- [Hardware][Intel-Gaudi] Update hpu-extension and update bucketing system for HPU device (#17186) · c48334d4
  Agata Dobrzyniewicz authored Apr 26, 2025
```
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
```
  c48334d4
- Allocate kv_cache with stride order (#16605) · 9e96f56e
  Shu Wang authored Apr 26, 2025
```
Signed-off-by: shuw <shuw@nvidia.com>
```
  9e96f56e
25 Apr, 2025 1 commit

[Quantization][FP8] Add support for FP8 models with input_scale for output... · a41351f3

rasmith authored Apr 25, 2025


[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

a41351f3

23 Apr, 2025 1 commit
- [Bugfix] Triton FA function takes no keyword arguments (#16902) · 047797ef
  vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  047797ef
22 Apr, 2025 3 commits

[FEAT][ROCm]: Support AITER MLA (#15893) · 30bc3e0f

vllmellm authored Apr 23, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>

30bc3e0f

[BugFix] Pass in correct VLLM config in FlashInfer backend (#13207) (#16973) · f961d7f6
Zhengyuan Su (苏政渊) authored Apr 22, 2025
```
Signed-off-by: 苏政渊 <suzhengyuan@moonshot.cn>
Co-authored-by: 苏政渊 <suzhengyuan@moonshot.cn>
```
f961d7f6

[FEAT][ROCm] Integrate Paged Attention Kernel from AITER (#15001) · 0e237f00

vllmellm authored Apr 22, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>

0e237f00

21 Apr, 2025 1 commit
- [XPU][Bugfix] minor fix for XPU (#15591) · fe3462c7
  Yan Ma authored Apr 22, 2025
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  fe3462c7
18 Apr, 2025 1 commit
- [ROCm] [Attention] Cleanup ROCm output passing (#16431) · aaec845f
  Luka Govedič authored Apr 18, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
```
  aaec845f
17 Apr, 2025 1 commit
- [Attention] Update to lastest FA3 code (#13111) · 183dad7a
  Lucas Wilkinson authored Apr 17, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  183dad7a
11 Apr, 2025 1 commit
- [Kernel] support merge_attn_states CUDA kernel, 3x speedup (#16173) · e9528f6d
  DefTruth authored Apr 11, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  e9528f6d
09 Apr, 2025 3 commits

[BugFix] fix some typos found by typos. (#16314) · 04149cce
yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
04149cce
[BugFix] logger is not callable (#16312) · 819d548e
yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
819d548e

[Bug] [ROCm] Fix Llama 4 Enablement Bug on ROCm: V0 ROCmFlashAttentionImpl and... · 2976dc27

TJian authored Apr 09, 2025


[Bug] [ROCm] Fix Llama 4 Enablement Bug on ROCm: V0 ROCmFlashAttentionImpl and Triton Fused MoE bugs (#16198)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>

2976dc27

08 Apr, 2025 1 commit
- Add warning for Attention backends that do not support irope yet (#16212) · 05a015d6
  Yong Hoon Shin authored Apr 07, 2025
  
  05a015d6
06 Apr, 2025 1 commit
- [Model] fix model testing for TeleChat2ForCausalLM and V0 llama4 (#16112) · 620fc2d0
  Lucia Fang authored Apr 05, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  620fc2d0
03 Apr, 2025 1 commit

[ROCM][KERNEL] Paged attention for V1 (#15720) · e73ff24e

Aleksandr Malyshev authored Apr 02, 2025


Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>

e73ff24e

27 Mar, 2025 1 commit
- [ROCm] Env variable to trigger custom PA (#15557) · ecff8309
  Gregory Shtrasberg authored Mar 27, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  ecff8309
26 Mar, 2025 1 commit
- Apply torchfix (#15532) · 1aa162e0
  cyyever authored Mar 26, 2025
```
Signed-off-by: cyy <cyyever@outlook.com>
```
  1aa162e0
25 Mar, 2025 1 commit
- [Kernel][CPU] CPU MLA (#14744) · 4f044b1d
  Thien Tran authored Mar 25, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  4f044b1d
23 Mar, 2025 1 commit

[V1] Enable V1 Fp8 cache for FA3 in the oracle (#15191) · dccf535f

Lucas Wilkinson authored Mar 23, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

dccf535f

22 Mar, 2025 1 commit
- Remove openvino support in favor of external plugin (#15339) · b877031d
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b877031d
21 Mar, 2025 1 commit
- [Bugfix] detect alibi and revert to FA2 (#15231) · 0032903a
  Travis Johnson authored Mar 20, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0032903a
20 Mar, 2025 1 commit
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
15 Mar, 2025 1 commit
- [CPU] Support FP8 KV cache (#14741) · a2ae4965
  Li, Jiang authored Mar 15, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a2ae4965
14 Mar, 2025 1 commit
- [Attention] MLA get rid of materialization (#14770) · 9532c498
  Lucas Wilkinson authored Mar 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9532c498
12 Mar, 2025 1 commit

[FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash... · 916836bb

TJian authored Mar 13, 2025


[FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash Attention to enable embedding models. (#14664)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>

916836bb