Commits · 92dd5703ecdb99aa4a4aee9817f28557907403a2 · gaoqiong / flash-attention

14 Nov, 2023 1 commit
- [LayerNorm] Add postnorm residual + LayerNorm/RMSNorm in Triton · 01771645
  Tri Dao authored Nov 13, 2023
  
  01771645
13 Nov, 2023 1 commit
- [LayerNorm] Implement residual + LayerNorm/RMSNorm in Triton · 79bd1a2d
  Tri Dao authored Nov 13, 2023
  
  79bd1a2d
19 Aug, 2023 1 commit
- Run isort and black on test files · 0e8c46ae
  Tri Dao authored Aug 18, 2023
  
  0e8c46ae
23 Jul, 2023 1 commit
- [LayerNorm] Add test for randomness · 2a2a3c4b
  Tri Dao authored Jul 23, 2023
  
  2a2a3c4b
04 Jul, 2023 1 commit
- [LayerNorm] Make sure memory addresses are aligned to 16 bytes · d2f4324f
  Tri Dao authored Jul 04, 2023
  
  d2f4324f
31 Mar, 2023 1 commit
- [LayerNorm] Implement LN with parallel residual, support dim 8k · 393882bc
  Tri Dao authored Mar 29, 2023
  
  393882bc
18 Jan, 2023 1 commit
- [FusedDense] Support relu, rename FusedDenseGeluDense -> FusedMLP · 88173a1a
  Tri Dao authored Jan 17, 2023
  
  88173a1a
07 Jan, 2023 2 commits
- [TP] Implement TensorParallel without sequence parallel · 93383bd5
  Tri Dao authored Jan 07, 2023
  
  93383bd5
- [LayerNorm] Implement RMS Norm · 6738d947
  Tri Dao authored Jan 06, 2023
  
  6738d947
24 Dec, 2022 1 commit
- Implement TensorParallel for FusedDense and FusedDenseGeluDense · 226a1b72
  Tri Dao authored Dec 23, 2022
  
  226a1b72
23 Dec, 2022 1 commit
- Simplify FusedDense · e68ebbe8
  Tri Dao authored Dec 22, 2022
  
  e68ebbe8
13 Dec, 2022 1 commit
- [LayerNorm] Support taking subset of input or subset of output · 5db33051
  Tri Dao authored Dec 12, 2022
  
  5db33051
11 Dec, 2022 1 commit
- [LayerNorm] Fuse LayerScale · ae137ed1
  Tri Dao authored Dec 10, 2022
  
  ae137ed1
09 Dec, 2022 1 commit
- [LayerNorm] Support all dimensions up to 6k (if divisible by 8) · 8c6609ae
  Tri Dao authored Dec 08, 2022
  
  8c6609ae
14 Nov, 2022 1 commit
- Add fused_dense and dropout_add_layernorm CUDA extensions · fa6d1ce4
  Tri Dao authored Nov 13, 2022
  
  fa6d1ce4