Commits · cb0daccc414021309b8748cbbcbfee5b2604eaf5 · gaoqiong / flash-attention

17 Aug, 2023 2 commits
- [FusedDense] Allow Row/ColumnParallelLinear to have uneven split · cb0daccc
  Tri Dao authored Aug 16, 2023
  
  cb0daccc
- [FusedDense] Run black on fused_dense.py · bcfa7c97
  Tri Dao authored Aug 16, 2023
  
  bcfa7c97
18 Apr, 2023 1 commit
- Implement GatedMlp · b630aef5
  Tri Dao authored Apr 18, 2023
  
  b630aef5
13 Apr, 2023 1 commit
- [FusedDense] Enable sqrelu activation in FusedMLP · 6f6e9a9a
  Tri Dao authored Apr 13, 2023
  
  6f6e9a9a
15 Mar, 2023 1 commit
- Support H100 for other CUDA extensions · dc08ea1c
  Tri Dao authored Mar 15, 2023
  
  dc08ea1c
18 Jan, 2023 1 commit
- [FusedDense] Support relu, rename FusedDenseGeluDense -> FusedMLP · 88173a1a
  Tri Dao authored Jan 17, 2023
  
  88173a1a
07 Jan, 2023 1 commit
- [TP] Implement TensorParallel without sequence parallel · 93383bd5
  Tri Dao authored Jan 07, 2023
  
  93383bd5
02 Jan, 2023 1 commit
- [FusedDense] Limit matrix dims to 2M (instead of 64k) · 1ec09ebd
  Tri Dao authored Jan 01, 2023
  
  1ec09ebd
01 Jan, 2023 1 commit
- [FusedDense] Kick off input all_gather before weight dtype conversion · 65b4064b
  Tri Dao authored Dec 31, 2022
  
  65b4064b
25 Dec, 2022 1 commit
- Implement Tensor Parallel for transformer Block · a8cfe515
  Tri Dao authored Dec 25, 2022
  
  a8cfe515
24 Dec, 2022 1 commit
- Implement TensorParallel for FusedDense and FusedDenseGeluDense · 226a1b72
  Tri Dao authored Dec 23, 2022
  
  226a1b72
23 Dec, 2022 1 commit
- Simplify FusedDense · e68ebbe8
  Tri Dao authored Dec 22, 2022
  
  e68ebbe8
14 Nov, 2022 2 commits
- Add MLP, MHA, Block, Embedding modules · d4b320b3
  Tri Dao authored Nov 13, 2022
  
  d4b320b3
- Add fused_dense and dropout_add_layernorm CUDA extensions · fa6d1ce4
  Tri Dao authored Nov 13, 2022
  
  fa6d1ce4