Commits · e279bf8ed967c7c3e62380e5ff7f69f4d343275f · gaoqiong / flash-attention

03 Oct, 2023 1 commit
- [Gen] Accept cache_batch_idx to index into the KV cache · e279bf8e
  Tri Dao authored Oct 03, 2023
  
  e279bf8e
26 Sep, 2023 1 commit
- Implement local attention · 083e8f52
  Tri Dao authored Sep 24, 2023
```
Co-authored-by: Timothee Lacroix <t@mistral.ai>
```
  083e8f52
21 Sep, 2023 1 commit
- Swap seqlen_q and ngroups when seqlen_q=1 (h/t Daniel Haziza) · 2d8ea9a5
  Tri Dao authored Sep 20, 2023
  
  2d8ea9a5
16 Sep, 2023 1 commit
- Implement rotary embedding in flash_attn_with_kvcache · ccbb14f3
  Tri Dao authored Sep 16, 2023
  
  ccbb14f3
13 Sep, 2023 1 commit
- Simplify the implementation of KVcache attn by appending KV first · 56b7fc6e
  Tri Dao authored Sep 13, 2023
  
  56b7fc6e
12 Sep, 2023 1 commit
- Remove some unused headers · bb9beb36
  Tri Dao authored Sep 12, 2023
  
  bb9beb36
11 Sep, 2023 1 commit
- Swap seqlen_q and nheads for MQA to speed it up (h/t Daniel Haziza) · ee77b931
  Tri Dao authored Sep 10, 2023
  
  ee77b931
04 Sep, 2023 2 commits
- Implement flash_attn_with_kvcache · 37c6e054
  Tri Dao authored Sep 04, 2023
  
  37c6e054
- Bump to v2.1.2 · 1dc1b6c8
  Tri Dao authored Sep 03, 2023
  
  1dc1b6c8
03 Sep, 2023 1 commit
- Fix splitKV combine function when local LSEs are all -inf · 26d7d92f
  Tri Dao authored Sep 03, 2023
  
  26d7d92f
30 Aug, 2023 1 commit
- Fix typo with lse_max == -INFINITY · 31920dda
  Tri Dao authored Aug 29, 2023
  
  31920dda
29 Aug, 2023 1 commit
- Implement splitKV attention · b1fbbd83
  Tri Dao authored Aug 29, 2023
  
  b1fbbd83
25 Aug, 2023 1 commit
- Change causal mask to be aligned to bottom-right instead of top-left · 9e5e8bc9
  Tri Dao authored Aug 21, 2023
  
  9e5e8bc9
24 Aug, 2023 1 commit
- Support flash attention 2 with causal masking when KV's seq length is longer... · e07aa036
  BoxiangW authored Aug 24, 2023
```
Support flash attention 2 with causal masking when KV's seq length is longer than Q's seq length. (#436)
```
  e07aa036
13 Aug, 2023 1 commit
- Update to Cutlass 3.1 · 3524e13c
  Tri Dao authored Aug 13, 2023
  
  3524e13c
27 Jul, 2023 1 commit

Kirthi Shankar Sivamani authored Jul 27, 2023



* Add RNG state to kernel launch params
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Save seed and offset for backward
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Single thread write to global mem
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* compute_dq_dk_dv_1colblock get seed and offset from launch params
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* compute_dq_dk_dv_1rowblock get seed and offset from launch params
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Change forward c++ APIs to save RNG state for backward
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Change backward c++ APIs to set RNG state for bprop launcher
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Bug fixes
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Python side API changes
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Bug fix; only save seeds instead of full offset
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Account for 3D grid size
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

---------
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

a03f6f8e

17 Jul, 2023 1 commit
- FlashAttention-2 release · 4f285b35
  Tri Dao authored Jul 17, 2023
  
  4f285b35