Commits · b17c6fe235b29c091f154c00c526dffa7ec4cce8 · gaoqiong / flash-attention

03 Jun, 2022 2 commits
- Reduce smem usage for Q and dO in the backward pass · b17c6fe2
  Tri Dao authored Jun 03, 2022
```
From 4KB per buffer to 2KB per buffer. This saves us 8KB of smem (each Q and dO
have 2 buffers)
```
  b17c6fe2
- Support Turing mma instructions · 2712aa4c
  Tri Dao authored Jun 02, 2022
  
  2712aa4c
02 Jun, 2022 7 commits
- Remove softmax fp16 max · 05087332
  Tri Dao authored Jun 02, 2022
  
  05087332
- Use Cutlass gemm as WarpMma · 14dc326e
  Tri Dao authored Jun 02, 2022
  
  14dc326e
- Remove old backward · e78e7c95
  Tri Dao authored Jun 02, 2022
  
  e78e7c95
- Add Cutlass as submodule · 512c98ee
  Tri Dao authored Jun 02, 2022
  
  512c98ee
- 3090 speedup · ad6c694b
  Dan Fu authored Jun 01, 2022
  
  ad6c694b
- Rename src -> flash_attn · 5a61cb77
  Tri Dao authored Jun 01, 2022
  
  5a61cb77
- Support SM86 GPUs · c41479d6
  Tri Dao authored Jun 01, 2022
  
  c41479d6
30 May, 2022 4 commits
- Citation · 4b7cfb5f
  Dan Fu authored May 30, 2022
  
  4b7cfb5f
- Jpg resolution · 963173fc
  Dan Fu authored May 30, 2022
  
  963173fc
- Fix jpg · cd04d298
  Dan Fu authored May 30, 2022
  
  cd04d298
- Add paper arXiv link · a7874518
  Tri Dao authored May 29, 2022
  
  a7874518
29 May, 2022 3 commits
- Edit roadmap · d9fff84b
  Tri Dao authored May 29, 2022
  
  d9fff84b
- Convert banner figure from pdf to jpg · e4ffe5d5
  Tri Dao authored May 29, 2022
  
  e4ffe5d5
- Reorganize directories, add banner figure · 67c37795
  Tri Dao authored May 29, 2022
  
  67c37795
28 May, 2022 1 commit
- Make png images into jpg for dark mode · 7025a092
  Dan Fu authored May 28, 2022
  
  7025a092
27 May, 2022 2 commits
- README typo · 4decc3c1
  Dan Fu authored May 27, 2022
  
  4decc3c1
- Add speedup to README · dc6d1300
  Dan Fu authored May 27, 2022
```
Update images

Update images

Update description
```
  dc6d1300
26 May, 2022 1 commit
- Rename, add benchmarking script · 9dbc491a
  Tri Dao authored May 26, 2022
  
  9dbc491a
20 May, 2022 1 commit
- First release · 1fcbe6f0
  Tri Dao authored May 20, 2022
  
  1fcbe6f0