Commits · abbc1311731867310635f9edc2a9ec18317c8c48 · gaoqiong / flash-attention

05 Jan, 2024 1 commit
- [LayerNorm] Switch from CUDA to Triton implementation · abbc1311
  Tri Dao authored Jan 05, 2024
  
  abbc1311
31 Mar, 2023 1 commit
- [LayerNorm] Implement LN with parallel residual, support dim 8k · 393882bc
  Tri Dao authored Mar 29, 2023
  
  393882bc
07 Jan, 2023 1 commit
- [LayerNorm] Implement RMS Norm · 6738d947
  Tri Dao authored Jan 06, 2023
  
  6738d947
09 Dec, 2022 1 commit
- [LayerNorm] Support all dimensions up to 6k (if divisible by 8) · 8c6609ae
  Tri Dao authored Dec 08, 2022
  
  8c6609ae
29 Nov, 2022 1 commit
- Release training code · 0bf5e500
  Tri Dao authored Nov 28, 2022
  
  0bf5e500
15 Nov, 2022 1 commit
- Mention that some CUDA extensions have only been tested on A100s · 43ab0b52
  Tri Dao authored Nov 15, 2022
  
  43ab0b52
14 Nov, 2022 2 commits
- Add GPT and ViT models · 2e33fc8e
  Tri Dao authored Nov 13, 2022
  
  2e33fc8e
- Add fused_dense and dropout_add_layernorm CUDA extensions · fa6d1ce4
  Tri Dao authored Nov 13, 2022
  
  fa6d1ce4