Commits · 344c988d1dcea878e65eb8cfb637eae621fc65b7 · gaoqiong / flash-attention

01 Aug, 2024 1 commit
- Sync with FA v2.6.0 to support soft capping (#13) · d562aa63
  Woosuk Kwon authored Jul 31, 2024
  
  d562aa63
26 Mar, 2024 7 commits
- allow smaller page sizes in varlen api · 79681482
  skrider authored Mar 26, 2024
  
  79681482
- add test for page table overflow · a63157ea
  skrider authored Mar 26, 2024
  
  a63157ea
- reshape rotary sin/cos copy to align with paged KV copy · 0f5a45ea
  skrider authored Feb 26, 2024
  
  0f5a45ea
- add page size 16 to tests · bc668587
  skrider authored Feb 13, 2024
  
  bc668587
- all working except rotary embedding · bde5aec8
  skrider authored Feb 13, 2024
  
  bde5aec8
- allow small page sizes in flash api · c05b8570
  skrider authored Feb 11, 2024
  
  c05b8570
- only test trivial block size · 409431b8
  skrider authored Feb 11, 2024
  
  409431b8
15 Mar, 2024 1 commit
- Enable paged attention in varlen forward (#831) · 2a15840f
  Grigory Sizov authored Mar 15, 2024
```
* Enable paged attention in varlen forward

* Format + fix padding
```
  2a15840f
21 Feb, 2024 1 commit
- Enable headdim 256 backward on consumer GPUs (Ampere, Ada) · 2406f288
  Tri Dao authored Feb 21, 2024
  
  2406f288
23 Jan, 2024 1 commit
- Implement page KV cache · 54e80a38
  Tri Dao authored Jan 22, 2024
```
Co-authored-by: ljss <450993438@qq.com>
```
  54e80a38
14 Jan, 2024 1 commit
- apply_dropout now takes tensor of rowcol layout · 10dad612
  Tri Dao authored Jan 14, 2024
  
  10dad612
13 Jan, 2024 1 commit
- Simplify writing softmax to gmem · a7b66ae2
  Tri Dao authored Jan 13, 2024
  
  a7b66ae2
24 Dec, 2023 1 commit
- Implement deterministic backward (thanks to Meituan) · 73265458
  Tri Dao authored Dec 23, 2023
  
  73265458
22 Dec, 2023 1 commit
- Clean up alibi, implement non-causal alibi · 5ab9b366
  Tri Dao authored Dec 21, 2023
  
  5ab9b366
03 Oct, 2023 1 commit
- [Gen] Accept cache_batch_idx to index into the KV cache · e279bf8e
  Tri Dao authored Oct 03, 2023
  
  e279bf8e
26 Sep, 2023 1 commit
- Implement local attention · 083e8f52
  Tri Dao authored Sep 24, 2023
```
Co-authored-by: Timothee Lacroix <t@mistral.ai>
```
  083e8f52
24 Sep, 2023 1 commit
- Don't over-allocate dq_accum in case of varlen · 65c234ed
  Tri Dao authored Sep 24, 2023
  
  65c234ed
21 Sep, 2023 1 commit
- Swap seqlen_q and ngroups when seqlen_q=1 (h/t Daniel Haziza) · 2d8ea9a5
  Tri Dao authored Sep 20, 2023
  
  2d8ea9a5
18 Sep, 2023 1 commit
- Swap seqlen_q, nheads for MQA when seqlen_q=1 for fwd (h/t Daniel H) · 3250ff3d
  Tri Dao authored Sep 18, 2023
  
  3250ff3d
16 Sep, 2023 1 commit
- Implement rotary embedding in flash_attn_with_kvcache · ccbb14f3
  Tri Dao authored Sep 16, 2023
  
  ccbb14f3
13 Sep, 2023 1 commit
- Simplify the implementation of KVcache attn by appending KV first · 56b7fc6e
  Tri Dao authored Sep 13, 2023
  
  56b7fc6e
04 Sep, 2023 2 commits
- Implement flash_attn_with_kvcache · 37c6e054
  Tri Dao authored Sep 04, 2023
  
  37c6e054
- Require CUDA 11.6+, clean up setup.py · 0c04943f
  Tri Dao authored Sep 03, 2023
  
  0c04943f
29 Aug, 2023 1 commit
- Implement splitKV attention · b1fbbd83
  Tri Dao authored Aug 29, 2023
  
  b1fbbd83
25 Aug, 2023 1 commit
- Change causal mask to be aligned to bottom-right instead of top-left · 9e5e8bc9
  Tri Dao authored Aug 21, 2023
  
  9e5e8bc9
19 Aug, 2023 1 commit
- Run isort and black on test files · 0e8c46ae
  Tri Dao authored Aug 18, 2023
  
  0e8c46ae
16 Aug, 2023 1 commit
- Fix Bwd NaN for varlen when seqlen_q >> seqlen_k and causal · c65b5106
  Tri Dao authored Aug 16, 2023
  
  c65b5106
13 Aug, 2023 1 commit
- Update to Cutlass 3.1 · 3524e13c
  Tri Dao authored Aug 13, 2023
  
  3524e13c
01 Aug, 2023 2 commits
- Fix race condition in bwd (overwriting sK) · 1c41d2b0
  Tri Dao authored Aug 01, 2023
  
  1c41d2b0
- Fix masking of bwd when seqlen is not divisible by 128 · a4f148b6
  Tri Dao authored Jul 31, 2023
  
  a4f148b6
17 Jul, 2023 1 commit
- FlashAttention-2 release · 4f285b35
  Tri Dao authored Jul 17, 2023
  
  4f285b35
13 Nov, 2022 2 commits
- Skip flash_attn_split test · a8fec99a
  Tri Dao authored Nov 13, 2022
  
  a8fec99a
- Don't enforce bitwise consistency for dq in race condition test · 9d3116ad
  Tri Dao authored Nov 13, 2022
```
Since we could be parallelizing over seqlen_k
```
  9d3116ad
09 Nov, 2022 1 commit
- Fix out-of-bound memory read · 6998e0ec
  Tri Dao authored Nov 09, 2022
  
  6998e0ec
06 Nov, 2022 1 commit
- Fix pipelining bug in Triton bwd with bias_type=matrix · 74797571
  Tri Dao authored Nov 06, 2022
  
  74797571
05 Nov, 2022 1 commit
- Parallelize CUDA bwd along seqlen_k instead of seqlen_q · 55778193
  Tri Dao authored Nov 05, 2022
```
This is faster since we only need to do atomic adds on dq, instead of atomic
adds on both dk and dv.
```
  55778193
04 Nov, 2022 2 commits
- Fix race condition in Triton bwd when there's bias · ff78ea41
  Tri Dao authored Nov 04, 2022
  
  ff78ea41
- Implement attention bias for Triton version · 86862cfd
  Tri Dao authored Nov 04, 2022
  
  86862cfd
02 Nov, 2022 1 commit
- Fix race condition in Triton bwd for non-po2 headdims · aacc10fb
  Tri Dao authored Nov 02, 2022
  
  aacc10fb