Commits · 1a2c3e8c25251fa30ebee074c27ecbf69c2bad2b · gaoqiong / flash-attention

25 Dec, 2023 1 commit
- Pass alibi slopes to flash_attn_with_kvcache during generation · 3f7d5786
  Tri Dao authored Dec 24, 2023
  
  3f7d5786
22 Dec, 2023 1 commit
- Add Alibi to MHA, test with Baichuan-13B · c3b21966
  Tri Dao authored Dec 21, 2023
  
  c3b21966
21 Sep, 2023 1 commit
- [Llama] Fix some tests, add tests for Llama 2 and CodeLlama · 0705d271
  Tri Dao authored Sep 20, 2023
  
  0705d271
20 Sep, 2023 1 commit
- [Gen] Rename max_sequence_len->max_seqlen, sequence_len_offset->seqlen_offset · e6a80264
  Tri Dao authored Sep 18, 2023
  
  e6a80264
18 Sep, 2023 1 commit
- [Gen] Don't use ft_attention, use flash_attn_with_kvcache instead · dfe29f5e
  Tri Dao authored Sep 18, 2023
  
  dfe29f5e
11 Sep, 2023 1 commit
- [Gen] Fix calling update_graph_cache in tests · 8a733cbd
  Tri Dao authored Sep 10, 2023
  
  8a733cbd
07 Sep, 2023 1 commit
- [Gen] Use flash_attn_with_kvcache in generation · a86442f0
  Tri Dao authored Sep 07, 2023
  
  a86442f0
04 Sep, 2023 1 commit
- Fix test_baichuan · 798858f9
  Tri Dao authored Sep 03, 2023
  
  798858f9
03 Sep, 2023 1 commit
- [Rotary] Pass max_seqlen from mha.py to rotary during inference · de2949f3
  Tri Dao authored Sep 03, 2023
  
  de2949f3
30 Aug, 2023 1 commit
- Support MQA + MP for decoding (#490) · 011ec323
  dan_the_3rd authored Aug 30, 2023
```
Co-authored-by: danthe3rd <danthe3rd>
```
  011ec323
26 Aug, 2023 1 commit
- Change causal for CrossAttention in mha.py to align to bottom right · a2974e85
  Tri Dao authored Aug 26, 2023
  
  a2974e85
18 Aug, 2023 2 commits

Run isort and black on python files · f1a73d07
Tri Dao authored Aug 18, 2023

f1a73d07

support when num_heads is not divisible by world_size; resolves #459 (#461) · bb4cded1

Xuechen Li authored Aug 18, 2023

* uneql rank.

* trim.

* enable passing in number of heads for each rank.

* simplify.

* simplify.

* cleanup.

* fix col parallel.

* fix bug with row parallel.

* fit out proj.

* refac.

* fix sharding logic.

* refac sharding.

* refac.

* support multiple of.

* make fn reuseable.

* fix bug in dimensions.

* scaffold.

* test uneven heads.

* fix test by adding barrier.

* refac.

* reuse code.

* clean up.

bb4cded1

17 Aug, 2023 1 commit
- [MHA] Run black on mha.py · bec5b3d3
  Tri Dao authored Aug 16, 2023
  
  bec5b3d3
23 Jul, 2023 1 commit
- [MHA] Implement MQA/GQA · 425dbcb6
  Tri Dao authored Jul 23, 2023
  
  425dbcb6
17 Jul, 2023 1 commit
- FlashAttention-2 release · 4f285b35
  Tri Dao authored Jul 17, 2023
  
  4f285b35
02 Jul, 2023 1 commit
- [Rotary] Make sure frequency calculation is in fp32 · 62e98144
  Tri Dao authored Jul 02, 2023
  
  62e98144
30 May, 2023 1 commit
- [Gen] Add rotary base as an argument to FT attention kernel · 48bc6eac
  Tri Dao authored May 30, 2023
  
  48bc6eac
21 Apr, 2023 2 commits
- [Gen] Move allocate_inference_cache to within the model · ba2fe7f3
  Tri Dao authored Apr 20, 2023
  
  ba2fe7f3
- [Gen] Fix FT kernel smem size, CG when batch size changed · 311d6606
  Tri Dao authored Apr 20, 2023
  
  311d6606
18 Apr, 2023 1 commit
- Have a separate nn.Dropout module in SelfAttention module · ac3b684c
  Tri Dao authored Apr 17, 2023
  
  ac3b684c
14 Apr, 2023 1 commit
- [Gen] Fix FT kernel when using CG · 605655bc
  Tri Dao authored Apr 14, 2023
  
  605655bc
29 Mar, 2023 1 commit
- [FT] Fix FT's single query attention for bf16 hdim128 rotary · f5d0fbd4
  Tri Dao authored Mar 28, 2023
  
  f5d0fbd4
22 Mar, 2023 1 commit
- Implement GPT-J · 4d87e4d8
  Tri Dao authored Mar 22, 2023
  
  4d87e4d8
16 Jan, 2023 1 commit
- [ViT] Support timm checkpoint, add tests · 780e8eea
  Tri Dao authored Jan 16, 2023
  
  780e8eea
15 Jan, 2023 1 commit
- [Gen] Make generation work with Tensor Parallel · 7c219154
  Tri Dao authored Jan 15, 2023
  
  7c219154
08 Jan, 2023 1 commit
- [Gen] Adjust shape of kv_cache when using FT · 0938298e
  Tri Dao authored Jan 07, 2023
  
  0938298e
07 Jan, 2023 3 commits
- [Gen] Test generation with rotary embedding · 11be742a
  Tri Dao authored Jan 07, 2023
  
  11be742a
- [TP] Implement TensorParallel without sequence parallel · 93383bd5
  Tri Dao authored Jan 07, 2023
  
  93383bd5
- fixed cross attention typeerror · aec35fd6
  Darius Lam authored Jan 07, 2023
  
  aec35fd6
04 Jan, 2023 1 commit
- [Gen] Add option to run generation with FT attention kernel · a668890f
  Tri Dao authored Jan 03, 2023
  
  a668890f
01 Jan, 2023 1 commit
- [FusedDense] Kick off input all_gather before weight dtype conversion · 65b4064b
  Tri Dao authored Dec 31, 2022
  
  65b4064b
28 Dec, 2022 2 commits
- Bump to v0.2.6 · a6ec1782
  Tri Dao authored Dec 27, 2022
  
  a6ec1782
- Implement generation for GPT · 63670fd8
  Tri Dao authored Dec 27, 2022
  
  63670fd8
25 Dec, 2022 1 commit
- Implement TensorParallel for MHA · 1e712ea8
  Tri Dao authored Dec 24, 2022
  
  1e712ea8
23 Dec, 2022 1 commit
- Simplify FusedDense · e68ebbe8
  Tri Dao authored Dec 22, 2022
  
  e68ebbe8
21 Dec, 2022 1 commit
- Implement XPos (Sun et al.) · 496e4f52
  Tri Dao authored Dec 21, 2022
  
  496e4f52
20 Dec, 2022 1 commit
- Implement last_layer_subset optimization for BERT · 13cdceb3
  Tri Dao authored Dec 19, 2022
  
  13cdceb3
19 Dec, 2022 1 commit
- Implement BERT · 5fb6df0e
  Tri Dao authored Dec 18, 2022
  
  5fb6df0e
14 Nov, 2022 1 commit
- Add MLP, MHA, Block, Embedding modules · d4b320b3
  Tri Dao authored Nov 13, 2022
  
  d4b320b3