Commits · 31f6b24fcff2b23c60e3c597a956f78a564f779a · OpenDAS / vllm_cscc

25 Mar, 2025 1 commit
- [Models]support blas and moe nn layout of deepseek-v3 · 89d1dd57
  zhuwenwen authored Mar 25, 2025
  
  89d1dd57
23 Mar, 2025 1 commit

[V1] Enable V1 Fp8 cache for FA3 in the oracle (#15191) · dccf535f

Lucas Wilkinson authored Mar 23, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

dccf535f

22 Mar, 2025 1 commit
- Remove openvino support in favor of external plugin (#15339) · b877031d
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b877031d
21 Mar, 2025 1 commit
- [Bugfix] detect alibi and revert to FA2 (#15231) · 0032903a
  Travis Johnson authored Mar 20, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0032903a
20 Mar, 2025 1 commit
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
19 Mar, 2025 1 commit
- skip fp8 kernel and _rocm_C extension · 9e053941
  zhuwenwen authored Mar 19, 2025
  
  9e053941
15 Mar, 2025 1 commit
- [CPU] Support FP8 KV cache (#14741) · a2ae4965
  Li, Jiang authored Mar 15, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a2ae4965
14 Mar, 2025 1 commit
- [Attention] MLA get rid of materialization (#14770) · 9532c498
  Lucas Wilkinson authored Mar 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9532c498
12 Mar, 2025 2 commits
- [FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash... · 916836bb
  TJian authored Mar 13, 2025
```
[FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash Attention to enable embedding models. (#14664)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  916836bb
- [ROCm] Enable chunked prefill/paged attention in MLA on ROCm (#14316) · d9f83d62
  Sage Moore authored Mar 12, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  d9f83d62
11 Mar, 2025 2 commits
- fix some typos : supported_head_sizes (#14627) · 53056731
  Yang.Tao authored Mar 12, 2025
  
  53056731
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
07 Mar, 2025 1 commit
- [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) · e1744502
  Luka Govedič authored Mar 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  e1744502
06 Mar, 2025 5 commits
- [V1][Bugfix] Standardize quantized kv cache rejection for attention backends (#14221) · 6832707e
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6832707e
- Fix mla prefill context performance (#13897) · 9f1710f1
  Ying Zhong authored Mar 07, 2025
```
Signed-off-by: ZhongYingMatrix <zhongyingmatrix@gmail.com>
```
  9f1710f1
- [Hardware] Update the flash attn tag to support Blackwell (#14244) · ed6ea065
  Pavani Majety authored Mar 05, 2025
  
  ed6ea065
- [Bugfix][CI] ALiBi test case in xformers multi_query_kv_attention (#11301) · 5ee10e99
  Nicolò Lucchesi authored Mar 06, 2025
  
  5ee10e99
- [BugFix] Fix prefix caching V0 MLA (#14255) · 4dacaa4a
  Lucas Wilkinson authored Mar 05, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Ying Zhong <zhongyingmatrix@gmail.com>
```
  4dacaa4a
03 Mar, 2025 1 commit
- [ROCm] Faster Custom Paged Attention kernels (#12348) · 848a6438
  TJian authored Mar 04, 2025
  
  848a6438
27 Feb, 2025 2 commits
- [core] Perf improvement for DSv3 on AMD GPUs (#13718) · 8294773e
  qli88 authored Feb 27, 2025
```
Signed-off-by: qli88 <qiang.li2@amd.com>
```
  8294773e
- [Kernel] FlashMLA integration (#13747) · f9590390
  Lucas Wilkinson authored Feb 26, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  f9590390
26 Feb, 2025 1 commit
- [ROCm] Disable chunked prefill/prefix caching when running MLA on non-cuda platforms (#13844) · 1d35662e
  Sage Moore authored Feb 25, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  1d35662e
25 Feb, 2025 3 commits
- [Bugfix] Initialize attention bias on the same device as Query/Key/Value (#13468) · 75e9d497
  Junlin Zhou authored Feb 25, 2025
  
  75e9d497
- [Bugfix] Support MLA for CompressedTensorsWNA16 (#13725) · 18e50593
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  18e50593
- [Misc] set single whitespace between log sentences (#13771) · 51010a18
  cjackal authored Feb 25, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  51010a18
23 Feb, 2025 1 commit
- [BugFix] Minor: logger import in attention backend (#13706) · 322d2a27
  Andy Lo authored Feb 23, 2025
```
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  322d2a27
22 Feb, 2025 2 commits
- [V1][Kernel] Refactor the prefix_prefill kernel so that the caller no longer... · 558db808
  Sage Moore authored Feb 22, 2025
```
[V1][Kernel] Refactor the prefix_prefill kernel so that the caller no longer has to pass in the context lengths (#13095)
```
  558db808
- [ROCM] fix native attention function call (#13650) · 68d630a0
  Gordon Wong authored Feb 22, 2025
  
  68d630a0
21 Feb, 2025 1 commit

[Attention] MLA with chunked prefill (#12639) · 288cc6c2

Lucas Wilkinson authored Feb 21, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

288cc6c2

20 Feb, 2025 1 commit
- [ROCm] MI300A compile targets deprecation (#13560) · 0023cd2b
  Gregory Shtrasberg authored Feb 20, 2025
  
  0023cd2b
19 Feb, 2025 1 commit

[Hardware][Gaudi][Feature] Support Contiguous Cache Fetch (#12139) · d0a7a276

Yu-Zhou authored Feb 19, 2025


Signed-off-by: yuzhou <yuzhou@habana.ai>
Signed-off-by: zhouyu5 <yu.zhou@intel.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

d0a7a276

15 Feb, 2025 1 commit
- [Bugfix] Massage MLA's usage of flash attn for RoCM (#13310) · 97a3d6d9
  Tyler Michael Smith authored Feb 15, 2025
  
  97a3d6d9
14 Feb, 2025 1 commit
- Expand MLA to support most types of quantization (#13181) · f0b2da72
  Michael Goin authored Feb 14, 2025
  
  f0b2da72
12 Feb, 2025 1 commit
- [Model] IBM/NASA Prithvi Geospatial model (#12830) · 974dfd49
  Christian Pinto authored Feb 12, 2025
  
  974dfd49
08 Feb, 2025 1 commit
- [bugfix] fix early import of flash attention (#12959) · fe743b79
  youkaichao authored Feb 09, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  fe743b79
06 Feb, 2025 4 commits
- Add Bamba Model (#10909) · aff40457
  Yu Chin Fabian Lim authored Feb 07, 2025
```
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  aff40457
- [Bugfix] Fix unsupported FA version check for Turing GPU (#12828) · 8108ac84
  Isotr0py authored Feb 07, 2025
  
  8108ac84
- [Kernel] Make rotary_embedding ops more flexible with input shape (#12777) · 85ac82d2
  Isotr0py authored Feb 07, 2025
  
  85ac82d2
- [Attention] Use FA3 for MLA on Hopper (#12807) · c786e757
  Lucas Wilkinson authored Feb 06, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  c786e757
05 Feb, 2025 1 commit
- [Hardware][Intel-Gaudi] Enable FusedSDPA support for Intel Gaudi (HPU) · af8486de
  Sanju C Sudhakaran authored Feb 06, 2025
  
  af8486de