Commits · 5018ac6ac531aabdb05c8af1ba3d98a2235bdbde · gaoqiong / flash-attention

15 Mar, 2024 1 commit
- fix: cast the alibi slopes to torch.float32 (#846) · 6bbc5323
  Markus Krimmel authored Mar 15, 2024
  
  6bbc5323
10 Feb, 2024 1 commit
- Add window_size option to ParallelMHA · a190df01
  Tri Dao authored Feb 04, 2024
  
  a190df01
31 Jan, 2024 1 commit
- Add window_size option to MHA and GPT · ef0ed106
  Tri Dao authored Jan 31, 2024
  
  ef0ed106
05 Jan, 2024 1 commit
- [LayerNorm] Switch from CUDA to Triton implementation · abbc1311
  Tri Dao authored Jan 05, 2024
  
  abbc1311
03 Jan, 2024 1 commit
- Fix: implement deterministic backward in mha (#748) · 386e3911
  jiaxingli authored Jan 03, 2024
```
* fix deterministic

* fix deterministic
```
  386e3911
25 Dec, 2023 1 commit
- Pass alibi slopes to flash_attn_with_kvcache during generation · 3f7d5786
  Tri Dao authored Dec 24, 2023
  
  3f7d5786
22 Dec, 2023 1 commit
- Add Alibi to MHA, test with Baichuan-13B · c3b21966
  Tri Dao authored Dec 21, 2023
  
  c3b21966
21 Sep, 2023 1 commit
- [Llama] Fix some tests, add tests for Llama 2 and CodeLlama · 0705d271
  Tri Dao authored Sep 20, 2023
  
  0705d271
20 Sep, 2023 1 commit
- [Gen] Rename max_sequence_len->max_seqlen, sequence_len_offset->seqlen_offset · e6a80264
  Tri Dao authored Sep 18, 2023
  
  e6a80264
18 Sep, 2023 1 commit
- [Gen] Don't use ft_attention, use flash_attn_with_kvcache instead · dfe29f5e
  Tri Dao authored Sep 18, 2023
  
  dfe29f5e
11 Sep, 2023 1 commit
- [Gen] Fix calling update_graph_cache in tests · 8a733cbd
  Tri Dao authored Sep 10, 2023
  
  8a733cbd
07 Sep, 2023 1 commit
- [Gen] Use flash_attn_with_kvcache in generation · a86442f0
  Tri Dao authored Sep 07, 2023
  
  a86442f0
04 Sep, 2023 2 commits
- [MLP] Implement SwiGLU with torch jiterator · 3557e0bb
  Tri Dao authored Sep 04, 2023
  
  3557e0bb
- Fix test_baichuan · 798858f9
  Tri Dao authored Sep 03, 2023
  
  798858f9
03 Sep, 2023 1 commit
- [Rotary] Pass max_seqlen from mha.py to rotary during inference · de2949f3
  Tri Dao authored Sep 03, 2023
  
  de2949f3
30 Aug, 2023 1 commit
- Support MQA + MP for decoding (#490) · 011ec323
  dan_the_3rd authored Aug 30, 2023
```
Co-authored-by: danthe3rd <danthe3rd>
```
  011ec323
26 Aug, 2023 1 commit
- Change causal for CrossAttention in mha.py to align to bottom right · a2974e85
  Tri Dao authored Aug 26, 2023
  
  a2974e85
18 Aug, 2023 2 commits

Run isort and black on python files · f1a73d07
Tri Dao authored Aug 18, 2023

f1a73d07

support when num_heads is not divisible by world_size; resolves #459 (#461) · bb4cded1

Xuechen Li authored Aug 18, 2023

* uneql rank.

* trim.

* enable passing in number of heads for each rank.

* simplify.

* simplify.

* cleanup.

* fix col parallel.

* fix bug with row parallel.

* fit out proj.

* refac.

* fix sharding logic.

* refac sharding.

* refac.

* support multiple of.

* make fn reuseable.

* fix bug in dimensions.

* scaffold.

* test uneven heads.

* fix test by adding barrier.

* refac.

* reuse code.

* clean up.

bb4cded1

17 Aug, 2023 1 commit
- [MHA] Run black on mha.py · bec5b3d3
  Tri Dao authored Aug 16, 2023
  
  bec5b3d3
10 Aug, 2023 1 commit
- [MLP] Change the check for out_features being None · 364a5b4a
  Tri Dao authored Aug 10, 2023
  
  364a5b4a
26 Jul, 2023 2 commits
- [MLP] Edit ParallelGatedMlp · 4c98d0b4
  Tri Dao authored Jul 26, 2023
  
  4c98d0b4
- Implement ParallelGatedMlp (#251) · 8ee62efc
  Haodong Lyu authored Jul 27, 2023
  
  8ee62efc
23 Jul, 2023 2 commits
- [MHA] Implement MQA/GQA · 425dbcb6
  Tri Dao authored Jul 23, 2023
  
  425dbcb6
- [MLP] Add ParallelMLP · 75e334d4
  Tri Dao authored Jul 22, 2023
  
  75e334d4
22 Jul, 2023 1 commit
- [GPT] Enable FlashAttention for GPT-J · b3177dfa
  Tri Dao authored Jul 21, 2023
  
  b3177dfa
21 Jul, 2023 1 commit
- [Block] Re-enable DropPath · 6fc1e07d
  Tri Dao authored Jul 21, 2023
  
  6fc1e07d
17 Jul, 2023 1 commit
- FlashAttention-2 release · 4f285b35
  Tri Dao authored Jul 17, 2023
  
  4f285b35
02 Jul, 2023 1 commit
- [Rotary] Make sure frequency calculation is in fp32 · 62e98144
  Tri Dao authored Jul 02, 2023
  
  62e98144
02 Jun, 2023 1 commit
- Fix a bug · 8e44c0ee
  ljss authored Jun 02, 2023
  
  8e44c0ee
30 May, 2023 1 commit
- [Gen] Add rotary base as an argument to FT attention kernel · 48bc6eac
  Tri Dao authored May 30, 2023
  
  48bc6eac
06 May, 2023 1 commit
- [BugFix] cannot unpack non-iterable NoneType object · 3889ba16
  Federico Berto authored May 07, 2023
  
  3889ba16
21 Apr, 2023 2 commits
- [Gen] Move allocate_inference_cache to within the model · ba2fe7f3
  Tri Dao authored Apr 20, 2023
  
  ba2fe7f3
- [Gen] Fix FT kernel smem size, CG when batch size changed · 311d6606
  Tri Dao authored Apr 20, 2023
  
  311d6606
19 Apr, 2023 1 commit
- Implement LLaMa · 96d10f65
  Tri Dao authored Apr 18, 2023
  
  96d10f65
18 Apr, 2023 2 commits
- Implement GatedMlp · b630aef5
  Tri Dao authored Apr 18, 2023
  
  b630aef5
- Have a separate nn.Dropout module in SelfAttention module · ac3b684c
  Tri Dao authored Apr 17, 2023
  
  ac3b684c
14 Apr, 2023 1 commit
- [Gen] Fix FT kernel when using CG · 605655bc
  Tri Dao authored Apr 14, 2023
  
  605655bc
13 Apr, 2023 1 commit
- make mlp hidden_features defaults to 4*in_features · 8c424156
  Zhiyuan Chen authored Apr 13, 2023
  
  8c424156
31 Mar, 2023 1 commit
- [LayerNorm] Implement LN with parallel residual, support dim 8k · 393882bc
  Tri Dao authored Mar 29, 2023
  
  393882bc