Commits · 008951f1d94a29eaeee48d25dbfa11df1ba12413 · gaoqiong / flash-attention

"src/vscode:/vscode.git/clone" did not exist on "27c6f5d394086c28791eac5eb86533d11b29cb46"

31 Oct, 2022 5 commits
- Support all head dimensions up to 128 in the Triton fwd · 008951f1
  Tri Dao authored Oct 30, 2022
  
  008951f1
- Support arbitrary seqlens (both q & k) in Triton bwd · b910bf14
  Tri Dao authored Oct 30, 2022
  
  b910bf14
- Support arbitrary seqlen_k in Triton bwd · dc554693
  Tri Dao authored Oct 30, 2022
  
  dc554693
- Fix Triton fwd to support seqlen not multiples of 128 · d11341fd
  Tri Dao authored Oct 30, 2022
  
  d11341fd
- Implement FlashAttention in Triton · b0c0db81
  Tri Dao authored Oct 30, 2022
  
  b0c0db81
24 Oct, 2022 1 commit
- Support all head dims that are multiples of 8, up to 128 · 46fd2a20
  Tri Dao authored Oct 24, 2022
  
  46fd2a20
23 Oct, 2022 1 commit
- Split bwd on the seqlen_q dimension · a5a8806d
  Tri Dao authored Oct 23, 2022
  
  a5a8806d
21 Oct, 2022 1 commit
- Rework dropout to decouple forward and backward · 1aa6d7d9
  Tri Dao authored Oct 18, 2022
```
They don't have to have the same block size, number of threads, etc.
```
  1aa6d7d9
16 Oct, 2022 1 commit
- Fix #54: set device for multi-GPU case · 52fb4b72
  Tri Dao authored Oct 16, 2022
  
  52fb4b72
14 Oct, 2022 1 commit
- Implement attention kernel that splits the batch into two · 5badfb78
  Tri Dao authored Oct 13, 2022
  
  5badfb78
05 Oct, 2022 1 commit
- Only run backward test for d=128 on A100 · 0c01568d
  Tri Dao authored Oct 04, 2022
  
  0c01568d
22 Jul, 2022 1 commit
- Add tests for numerical error · 2ed471ec
  Tri Dao authored Jul 22, 2022
  
  2ed471ec