Commits · 5018ac6ac531aabdb05c8af1ba3d98a2235bdbde · gaoqiong / flash-attention

22 Dec, 2023 1 commit
- Add Alibi to MHA, test with Baichuan-13B · c3b21966
  Tri Dao authored Dec 21, 2023
  
  c3b21966
04 Sep, 2023 1 commit
- [MLP] Implement SwiGLU with torch jiterator · 3557e0bb
  Tri Dao authored Sep 04, 2023
  
  3557e0bb
18 Aug, 2023 1 commit
- Run isort and black on python files · f1a73d07
  Tri Dao authored Aug 18, 2023
  
  f1a73d07
10 Aug, 2023 1 commit
- [MLP] Change the check for out_features being None · 364a5b4a
  Tri Dao authored Aug 10, 2023
  
  364a5b4a
26 Jul, 2023 2 commits
- [MLP] Edit ParallelGatedMlp · 4c98d0b4
  Tri Dao authored Jul 26, 2023
  
  4c98d0b4
- Implement ParallelGatedMlp (#251) · 8ee62efc
  Haodong Lyu authored Jul 27, 2023
  
  8ee62efc
23 Jul, 2023 1 commit
- [MLP] Add ParallelMLP · 75e334d4
  Tri Dao authored Jul 22, 2023
  
  75e334d4
19 Apr, 2023 1 commit
- Implement LLaMa · 96d10f65
  Tri Dao authored Apr 18, 2023
  
  96d10f65
18 Apr, 2023 1 commit
- Implement GatedMlp · b630aef5
  Tri Dao authored Apr 18, 2023
  
  b630aef5
13 Apr, 2023 1 commit
- make mlp hidden_features defaults to 4*in_features · 8c424156
  Zhiyuan Chen authored Apr 13, 2023
  
  8c424156
18 Jan, 2023 1 commit
- [FusedDense] Support relu, rename FusedDenseGeluDense -> FusedMLP · 88173a1a
  Tri Dao authored Jan 17, 2023
  
  88173a1a
24 Dec, 2022 1 commit
- Implement TensorParallel for FusedDense and FusedDenseGeluDense · 226a1b72
  Tri Dao authored Dec 23, 2022
  
  226a1b72
23 Dec, 2022 1 commit
- Simplify FusedDense · e68ebbe8
  Tri Dao authored Dec 22, 2022
  
  e68ebbe8
20 Dec, 2022 1 commit
- Implement last_layer_subset optimization for BERT · 13cdceb3
  Tri Dao authored Dec 19, 2022
  
  13cdceb3
23 Nov, 2022 1 commit
- [ViT] Use dropout_add_ln for the 1st layer norm · 1feb9426
  Tri Dao authored Nov 23, 2022
  
  1feb9426
14 Nov, 2022 1 commit
- Add MLP, MHA, Block, Embedding modules · d4b320b3
  Tri Dao authored Nov 13, 2022
  
  d4b320b3