Commits · dfc60f6b7d9a2bd94ffa95ae3f0f7d3c4fb4efcb · gaoqiong / flash-attention

19 Jul, 2023 1 commit
- [LayerNorm] Fix typo in ln_api.cpp · dfc60f6b
  Ikko Eltociear Ashimine authored Jul 20, 2023
```
unintialized -> uninitialized
```
  dfc60f6b
31 Mar, 2023 1 commit
- [LayerNorm] Implement LN with parallel residual, support dim 8k · 393882bc
  Tri Dao authored Mar 29, 2023
  
  393882bc
19 Jan, 2023 1 commit
- [LayerNorm] Rename x1 -> residual · eb33e587
  Tri Dao authored Jan 19, 2023
  
  eb33e587
07 Jan, 2023 1 commit
- [LayerNorm] Implement RMS Norm · 6738d947
  Tri Dao authored Jan 06, 2023
  
  6738d947
25 Dec, 2022 1 commit
- Implement Tensor Parallel for transformer Block · a8cfe515
  Tri Dao authored Dec 25, 2022
  
  a8cfe515
13 Dec, 2022 1 commit
- [LayerNorm] Support taking subset of input or subset of output · 5db33051
  Tri Dao authored Dec 12, 2022
  
  5db33051
11 Dec, 2022 1 commit
- [LayerNorm] Fuse LayerScale · ae137ed1
  Tri Dao authored Dec 10, 2022
  
  ae137ed1
09 Dec, 2022 1 commit
- [LayerNorm] Support all dimensions up to 6k (if divisible by 8) · 8c6609ae
  Tri Dao authored Dec 08, 2022
  
  8c6609ae
14 Nov, 2022 1 commit
- Add fused_dense and dropout_add_layernorm CUDA extensions · fa6d1ce4
  Tri Dao authored Nov 13, 2022
  
  fa6d1ce4