Commits · b32efb1a4d36a890c689a15eb0848922e11690b0 · gaoqiong / flash-attention

20 Feb, 2024 1 commit
- Don't need to reduce row_sum during online softmax · b32efb1a
  Tri Dao authored Feb 20, 2024
  
  b32efb1a
21 Jan, 2024 1 commit
- Change inline to __forceinline__, use __grid_constant__ param · ed4959b2
  Tri Dao authored Jan 20, 2024
  
  ed4959b2
20 Jan, 2024 1 commit
- Make Softmax an object · 6f706eff
  Tri Dao authored Jan 15, 2024
  
  6f706eff
14 Jan, 2024 4 commits
- Move softmax_rescale_o to softmax.h · df1418f9
  Tri Dao authored Jan 14, 2024
  
  df1418f9
- Move masking to a separate file (mask.h) · 6777336a
  Tri Dao authored Jan 14, 2024
  
  6777336a
- Move dropout to a separate file (dropout.h) · 1274ec3e
  Tri Dao authored Jan 14, 2024
  
  1274ec3e
- apply_dropout now takes tensor of rowcol layout · 10dad612
  Tri Dao authored Jan 14, 2024
  
  10dad612
22 Dec, 2023 1 commit
- Clean up alibi, implement non-causal alibi · 5ab9b366
  Tri Dao authored Dec 21, 2023
  
  5ab9b366
26 Sep, 2023 1 commit
- Implement local attention · 083e8f52
  Tri Dao authored Sep 24, 2023
```
Co-authored-by: Timothee Lacroix <t@mistral.ai>
```
  083e8f52
12 Sep, 2023 1 commit
- Remove some unused headers · bb9beb36
  Tri Dao authored Sep 12, 2023
  
  bb9beb36
25 Aug, 2023 1 commit
- Change causal mask to be aligned to bottom-right instead of top-left · 9e5e8bc9
  Tri Dao authored Aug 21, 2023
  
  9e5e8bc9
24 Aug, 2023 1 commit
- Support flash attention 2 with causal masking when KV's seq length is longer... · e07aa036
  BoxiangW authored Aug 24, 2023
```
Support flash attention 2 with causal masking when KV's seq length is longer than Q's seq length. (#436)
```
  e07aa036
01 Aug, 2023 1 commit
- Fix masking of bwd when seqlen is not divisible by 128 · a4f148b6
  Tri Dao authored Jul 31, 2023
  
  a4f148b6
17 Jul, 2023 1 commit
- FlashAttention-2 release · 4f285b35
  Tri Dao authored Jul 17, 2023
  
  4f285b35