Commits · 675ba75f40f756b113955f77c324f962da1830eb · OpenDAS / vllm_cscc

04 Apr, 2025 1 commit
- [ROCm][Bugfix] Use platform specific FP8 dtype (#15717) · 40a36ccf
  Gregory Shtrasberg authored Apr 04, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  40a36ccf
03 Apr, 2025 2 commits

[Neuron][kernel] Fuse kv cache into a single tensor (#15911) · d2b58ca2
Liangfu Chen authored Apr 03, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
d2b58ca2

[ROCM][KERNEL] Paged attention for V1 (#15720) · e73ff24e

Aleksandr Malyshev authored Apr 02, 2025


Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>

e73ff24e

27 Mar, 2025 1 commit
- [ROCm] Env variable to trigger custom PA (#15557) · ecff8309
  Gregory Shtrasberg authored Mar 27, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  ecff8309
26 Mar, 2025 2 commits

Apply torchfix (#15532) · 1aa162e0
cyyever authored Mar 26, 2025
```
Signed-off-by: cyy <cyyever@outlook.com>
```
1aa162e0

[BugFix] Fix nightly MLA failure (FA2 + MLA chunked prefill, i.e. V1,... · 33437bc6

Lucas Wilkinson authored Mar 25, 2025


[BugFix] Fix nightly MLA failure (FA2 + MLA chunked prefill, i.e. V1, producing bad results) (#15492)
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>

33437bc6

25 Mar, 2025 2 commits
- [Kernel][CPU] CPU MLA (#14744) · 4f044b1d
  Thien Tran authored Mar 25, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  4f044b1d
- [Models]support blas and moe nn layout of deepseek-v3 · 89d1dd57
  zhuwenwen authored Mar 25, 2025
  
  89d1dd57
23 Mar, 2025 1 commit

[V1] Enable V1 Fp8 cache for FA3 in the oracle (#15191) · dccf535f

Lucas Wilkinson authored Mar 23, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

dccf535f

22 Mar, 2025 1 commit
- Remove openvino support in favor of external plugin (#15339) · b877031d
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b877031d
21 Mar, 2025 2 commits
- [TPU][V1] MHA Pallas backend (#15288) · cfbb8c93
  Nicolò Lucchesi authored Mar 21, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  cfbb8c93
- [Bugfix] detect alibi and revert to FA2 (#15231) · 0032903a
  Travis Johnson authored Mar 20, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0032903a
20 Mar, 2025 1 commit
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
19 Mar, 2025 1 commit
- skip fp8 kernel and _rocm_C extension · 9e053941
  zhuwenwen authored Mar 19, 2025
  
  9e053941
15 Mar, 2025 1 commit
- [CPU] Support FP8 KV cache (#14741) · a2ae4965
  Li, Jiang authored Mar 15, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a2ae4965
14 Mar, 2025 2 commits

[Attention] MLA get rid of materialization (#14770) · 9532c498
Lucas Wilkinson authored Mar 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
9532c498

[Kernel] [V1] Further optimizations to ROCm (Triton) Backend to better handle GQA. (#14431) · fb4c7f8e

Thomas Parnell authored Mar 14, 2025


Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Jan van Lunteren <jvl@zurich.ibm.com>
Co-authored-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>

fb4c7f8e

12 Mar, 2025 3 commits
- [FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash... · 916836bb
  TJian authored Mar 13, 2025
```
[FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash Attention to enable embedding models. (#14664)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  916836bb
- [ROCm] Enable chunked prefill/paged attention in MLA on ROCm (#14316) · d9f83d62
  Sage Moore authored Mar 12, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  d9f83d62
- [CPU] Upgrade CPU backend to torch-2.6 (#13381) · ff47aab0
  Li, Jiang authored Mar 12, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  ff47aab0
11 Mar, 2025 3 commits
- fix some typos : supported_head_sizes (#14627) · 53056731
  Yang.Tao authored Mar 12, 2025
  
  53056731
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
- [neuron] add reshape_and_cache (#14391) · c91b64f7
  Liangfu Chen authored Mar 10, 2025
  
  c91b64f7
10 Mar, 2025 1 commit
- Correct capitalisation: `VLLM` -> `vLLM` (#14562) · 3b352a2f
  Harry Mellor authored Mar 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3b352a2f
07 Mar, 2025 1 commit
- [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) · e1744502
  Luka Govedič authored Mar 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  e1744502
06 Mar, 2025 8 commits
- [V1][Bugfix] Standardize quantized kv cache rejection for attention backends (#14221) · 6832707e
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6832707e
- [Bug] Fix Attention when ignored in by quant_method (#14313) · 6b2ef5cd
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6b2ef5cd
- Fix mla prefill context performance (#13897) · 9f1710f1
  Ying Zhong authored Mar 07, 2025
```
Signed-off-by: ZhongYingMatrix <zhongyingmatrix@gmail.com>
```
  9f1710f1
- Add authors to license header. (#14371) · e642ec96
  Thomas Parnell authored Mar 06, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Jan van Lunteren <jvl@zurich.ibm.com>
```
  e642ec96
- [Kernel] [V1] Improved performance for V1 Triton (ROCm) backend (#14152) · 6bd1dd9d
  Thomas Parnell authored Mar 06, 2025
  
  6bd1dd9d
- [Hardware] Update the flash attn tag to support Blackwell (#14244) · ed6ea065
  Pavani Majety authored Mar 05, 2025
  
  ed6ea065
- [Bugfix][CI] ALiBi test case in xformers multi_query_kv_attention (#11301) · 5ee10e99
  Nicolò Lucchesi authored Mar 06, 2025
  
  5ee10e99
- [BugFix] Fix prefix caching V0 MLA (#14255) · 4dacaa4a
  Lucas Wilkinson authored Mar 05, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Ying Zhong <zhongyingmatrix@gmail.com>
```
  4dacaa4a
05 Mar, 2025 1 commit
- [V1] EP/TP MoE + DP Attention (#13931) · 72c62eae
  Tyler Michael Smith authored Mar 05, 2025
  
  72c62eae
03 Mar, 2025 1 commit
- [ROCm] Faster Custom Paged Attention kernels (#12348) · 848a6438
  TJian authored Mar 04, 2025
  
  848a6438
27 Feb, 2025 3 commits
- [core] Perf improvement for DSv3 on AMD GPUs (#13718) · 8294773e
  qli88 authored Feb 27, 2025
```
Signed-off-by: qli88 <qiang.li2@amd.com>
```
  8294773e
- [Attention] MLA support for V1 (#13789) · 58d1b2aa
  Yang Chen authored Feb 27, 2025
```
Signed-off-by: Yang Chen <yangche@fb.com>
```
  58d1b2aa
- [Kernel] FlashMLA integration (#13747) · f9590390
  Lucas Wilkinson authored Feb 26, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  f9590390
26 Feb, 2025 2 commits
- Add comments on accessing `kv_cache` and `attn_metadata` (#13887) · 0ecdd980
  Harry Mellor authored Feb 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0ecdd980
- [ROCm] Disable chunked prefill/prefix caching when running MLA on non-cuda platforms (#13844) · 1d35662e
  Sage Moore authored Feb 25, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  1d35662e