Commits · dc08ea1c33afca500a3d4ada907608f7815a11d9 · gaoqiong / flash-attention

15 Mar, 2023 1 commit
- Support H100 for other CUDA extensions · dc08ea1c
  Tri Dao authored Mar 15, 2023
  
  dc08ea1c
18 Jan, 2023 1 commit
- [FusedDense] Support relu, rename FusedDenseGeluDense -> FusedMLP · 88173a1a
  Tri Dao authored Jan 17, 2023
  
  88173a1a
07 Jan, 2023 1 commit
- [TP] Implement TensorParallel without sequence parallel · 93383bd5
  Tri Dao authored Jan 07, 2023
  
  93383bd5
02 Jan, 2023 1 commit
- [FusedDense] Limit matrix dims to 2M (instead of 64k) · 1ec09ebd
  Tri Dao authored Jan 01, 2023
  
  1ec09ebd
01 Jan, 2023 1 commit
- [FusedDense] Kick off input all_gather before weight dtype conversion · 65b4064b
  Tri Dao authored Dec 31, 2022
  
  65b4064b
25 Dec, 2022 1 commit
- Implement Tensor Parallel for transformer Block · a8cfe515
  Tri Dao authored Dec 25, 2022
  
  a8cfe515
24 Dec, 2022 1 commit
- Implement TensorParallel for FusedDense and FusedDenseGeluDense · 226a1b72
  Tri Dao authored Dec 23, 2022
  
  226a1b72
23 Dec, 2022 1 commit
- Simplify FusedDense · e68ebbe8
  Tri Dao authored Dec 22, 2022
  
  e68ebbe8
14 Nov, 2022 2 commits
- Add MLP, MHA, Block, Embedding modules · d4b320b3
  Tri Dao authored Nov 13, 2022
  
  d4b320b3
- Add fused_dense and dropout_add_layernorm CUDA extensions · fa6d1ce4
  Tri Dao authored Nov 13, 2022
  
  fa6d1ce4