Commits · 46fd2a20b20849598b1abb438ea09520449d7eb3 · gaoqiong / flash-attention

24 Oct, 2022 1 commit
- Support all head dims that are multiples of 8, up to 128 · 46fd2a20
  Tri Dao authored Oct 24, 2022
  
  46fd2a20
22 Jul, 2022 1 commit
- Add tests for numerical error · 2ed471ec
  Tri Dao authored Jul 22, 2022
  
  2ed471ec
12 Jul, 2022 1 commit
- Edit mention of Triton implementation · 42f54d88
  Tri Dao authored Jul 11, 2022
```
Phil Tillet suggests calling it "experimental".
```
  42f54d88
11 Jul, 2022 1 commit
- Link to Triton implementation · 4577151f
  Tri Dao authored Jul 11, 2022
  
  4577151f
10 Jul, 2022 3 commits
- Link to IEEE Spectrum article on MLPerf · d1fc80a3
  Tri Dao authored Jul 10, 2022
  
  d1fc80a3
- Edit README to mention bf16 support · 1bbebccc
  Tri Dao authored Jul 09, 2022
  
  1bbebccc
- Implement for bf16 · de19de7a
  Tri Dao authored Jul 09, 2022
  
  de19de7a
04 Jul, 2022 1 commit
- Implement cross attention · 6c3a8c65
  Tri Dao authored Jun 30, 2022
  
  6c3a8c65
27 Jun, 2022 1 commit
- Add README section on issues · 450b64fe
  Tri Dao authored Jun 27, 2022
  
  450b64fe
14 Jun, 2022 2 commits
- More explanation · 765741c1
  Dan Fu authored Jun 14, 2022
  
  765741c1
- Speedup graph for A100, d128 · 2d5b2483
  Dan Fu authored Jun 14, 2022
  
  2d5b2483
12 Jun, 2022 1 commit
- Implement bwd for head dim 128 · d3e64409
  Tri Dao authored Jun 11, 2022
  
  d3e64409
05 Jun, 2022 2 commits
- Broken link · 0a398dfc
  Dan Fu authored Jun 04, 2022
  
  0a398dfc
- T4 · bd60750e
  Dan Fu authored Jun 04, 2022
  
  bd60750e
04 Jun, 2022 1 commit
- Edit README: support Turing (SM75) · f2d8d410
  Tri Dao authored Jun 04, 2022
  
  f2d8d410
02 Jun, 2022 3 commits
- 3090 speedup · ad6c694b
  Dan Fu authored Jun 01, 2022
  
  ad6c694b
- Rename src -> flash_attn · 5a61cb77
  Tri Dao authored Jun 01, 2022
  
  5a61cb77
- Support SM86 GPUs · c41479d6
  Tri Dao authored Jun 01, 2022
  
  c41479d6
30 May, 2022 2 commits
- Citation · 4b7cfb5f
  Dan Fu authored May 30, 2022
  
  4b7cfb5f
- Add paper arXiv link · a7874518
  Tri Dao authored May 29, 2022
  
  a7874518
29 May, 2022 3 commits
- Edit roadmap · d9fff84b
  Tri Dao authored May 29, 2022
  
  d9fff84b
- Convert banner figure from pdf to jpg · e4ffe5d5
  Tri Dao authored May 29, 2022
  
  e4ffe5d5
- Reorganize directories, add banner figure · 67c37795
  Tri Dao authored May 29, 2022
  
  67c37795
28 May, 2022 1 commit
- Make png images into jpg for dark mode · 7025a092
  Dan Fu authored May 28, 2022
  
  7025a092
27 May, 2022 2 commits
- README typo · 4decc3c1
  Dan Fu authored May 27, 2022
  
  4decc3c1
- Add speedup to README · dc6d1300
  Dan Fu authored May 27, 2022
```
Update images

Update images

Update description
```
  dc6d1300
26 May, 2022 1 commit
- Rename, add benchmarking script · 9dbc491a
  Tri Dao authored May 26, 2022
  
  9dbc491a
20 May, 2022 1 commit
- First release · 1fcbe6f0
  Tri Dao authored May 20, 2022
  
  1fcbe6f0