Commits · 0f5a45ea5303cae2f2acac1bf0201bb734561d0d · gaoqiong / flash-attention

26 Mar, 2024 2 commits
- reshape rotary sin/cos copy to align with paged KV copy · 0f5a45ea
  skrider authored Feb 26, 2024
  
  0f5a45ea
- reshape gmem copy · 14b190bc
  skrider authored Feb 11, 2024
  
  14b190bc
21 Feb, 2024 1 commit
- Enable headdim 256 backward on consumer GPUs (Ampere, Ada) · 2406f288
  Tri Dao authored Feb 21, 2024
  
  2406f288
23 Jan, 2024 1 commit
- Use int64_t instead of uint32_t in kernel_traits.h · 36bc29ed
  Tri Dao authored Jan 22, 2024
  
  36bc29ed
21 Jan, 2024 1 commit
- Update cutlass to v3.4.0 · 8f4d82cf
  Tri Dao authored Jan 20, 2024
  
  8f4d82cf
14 Jan, 2024 2 commits
- Move masking to a separate file (mask.h) · 6777336a
  Tri Dao authored Jan 14, 2024
  
  6777336a
- Remove seqq_parallel backward kernel that's not used · 9448264d
  Tri Dao authored Jan 14, 2024
  
  9448264d
13 Jan, 2024 1 commit
- Simplify writing softmax to gmem · a7b66ae2
  Tri Dao authored Jan 13, 2024
  
  a7b66ae2
12 Jan, 2024 1 commit
- Simplify SmemLayoutVtransposed in kernel_traits.h · 8d1b169e
  Tri Dao authored Jan 12, 2024
  
  8d1b169e
16 Sep, 2023 1 commit
- Implement rotary embedding in flash_attn_with_kvcache · ccbb14f3
  Tri Dao authored Sep 16, 2023
  
  ccbb14f3
04 Sep, 2023 1 commit
- Remove unused sdPsum in dot_do_o function · 5953c4f5
  Tri Dao authored Sep 03, 2023
  
  5953c4f5
01 Sep, 2023 1 commit
- Remove commented out code in bwd (#512) · 37e32feb
  Sophia Wisdom authored Sep 01, 2023
```
* Remove lots of comments

* Remove unused traits
```
  37e32feb
29 Aug, 2023 1 commit
- Implement splitKV attention · b1fbbd83
  Tri Dao authored Aug 29, 2023
  
  b1fbbd83
13 Aug, 2023 1 commit
- Prepare for Cutlass 3.2 · dbd79237
  Tri Dao authored Aug 13, 2023
  
  dbd79237
17 Jul, 2023 1 commit
- FlashAttention-2 release · 4f285b35
  Tri Dao authored Jul 17, 2023
  
  4f285b35