Commits · 7d25a4ec4f4ecef5df1a208ec74ca34455dcb439 · gaoqiong / flash-attention

13 Apr, 2023 1 commit
- Handle FlashAttnQKVPackedSplitFunc by making rng_state optional in backward · 7d25a4ec
  Kirthi Shankar Sivamani authored Apr 13, 2023
```
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
```
  7d25a4ec
12 Apr, 2023 1 commit
- Support CUDA graph capture · 31018c5f
  Kirthi Shankar Sivamani authored Apr 12, 2023
```
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
```
  31018c5f
07 Apr, 2023 1 commit
- [FusedDense] Set workspace size to 32M for Hopper and 4M for others · dec4f2e9
  Tri Dao authored Apr 06, 2023
  
  dec4f2e9
31 Mar, 2023 1 commit
- [LayerNorm] Implement LN with parallel residual, support dim 8k · 393882bc
  Tri Dao authored Mar 29, 2023
  
  393882bc
29 Mar, 2023 1 commit
- [FT] Fix FT's single query attention for bf16 hdim128 rotary · f5d0fbd4
  Tri Dao authored Mar 28, 2023
  
  f5d0fbd4
15 Mar, 2023 2 commits
- Support H100 for other CUDA extensions · dc08ea1c
  Tri Dao authored Mar 15, 2023
  
  dc08ea1c
- Support H100 · 1b18f1b7
  Tri Dao authored Mar 15, 2023
  
  1b18f1b7
14 Mar, 2023 1 commit
- [Rotary] Implement GPT-J style (interleaved) rotary · e45a46a5
  Tri Dao authored Mar 14, 2023
  
  e45a46a5
25 Jan, 2023 1 commit
- [FA] Remove unused variable rng_engine_inputs · 6b4a4821
  Tri Dao authored Jan 25, 2023
  
  6b4a4821
19 Jan, 2023 1 commit
- [LayerNorm] Rename x1 -> residual · eb33e587
  Tri Dao authored Jan 19, 2023
  
  eb33e587
18 Jan, 2023 1 commit
- [FusedDense] Support relu, rename FusedDenseGeluDense -> FusedMLP · 88173a1a
  Tri Dao authored Jan 17, 2023
  
  88173a1a
15 Jan, 2023 2 commits
- [Gen] Pass qkv_stride to ft_attention kernel for batched generation · f1e01c27
  Tri Dao authored Jan 15, 2023
  
  f1e01c27
- [Gen] Make generation work with Tensor Parallel · 7c219154
  Tri Dao authored Jan 15, 2023
  
  7c219154
07 Jan, 2023 1 commit
- [LayerNorm] Implement RMS Norm · 6738d947
  Tri Dao authored Jan 06, 2023
  
  6738d947
06 Jan, 2023 1 commit
- [Compilation] Change BOOL_SWITCH to fix Windows compilation · a1f49a2b
  Tri Dao authored Jan 06, 2023
```
Follow xFormers's DISTPATCH_BOOL. Haven't tested it on Windows.
```
  a1f49a2b
04 Jan, 2023 3 commits
- [Gen, FT] Use fp32 accum for FMA · be1afaa2
  Tri Dao authored Jan 03, 2023
  
  be1afaa2
- [Gen, FT] Use tlength instead of params.timestep for rotary · f266fc72
  Tri Dao authored Jan 03, 2023
  
  f266fc72
- [Gen] Add kernel from FasterTransformer for benchmarking · a01d1213
  Tri Dao authored Jan 03, 2023
  
  a01d1213
25 Dec, 2022 2 commits
- Implement Tensor Parallel for transformer Block · a8cfe515
  Tri Dao authored Dec 25, 2022
  
  a8cfe515
- Implement TensorParallel for MHA · 1e712ea8
  Tri Dao authored Dec 24, 2022
  
  1e712ea8
24 Dec, 2022 1 commit
- Implement TensorParallel for FusedDense and FusedDenseGeluDense · 226a1b72
  Tri Dao authored Dec 23, 2022
  
  226a1b72
23 Dec, 2022 2 commits
- Add smoothing for CrossEntropyParallel, rename to CrossEntropyLoss · dff68c2b
  Tri Dao authored Dec 23, 2022
  
  dff68c2b
- Simplify FusedDense · e68ebbe8
  Tri Dao authored Dec 22, 2022
  
  e68ebbe8
13 Dec, 2022 1 commit
- [LayerNorm] Support taking subset of input or subset of output · 5db33051
  Tri Dao authored Dec 12, 2022
  
  5db33051
11 Dec, 2022 1 commit
- [LayerNorm] Fuse LayerScale · ae137ed1
  Tri Dao authored Dec 10, 2022
  
  ae137ed1
09 Dec, 2022 1 commit
- [LayerNorm] Support all dimensions up to 6k (if divisible by 8) · 8c6609ae
  Tri Dao authored Dec 08, 2022
  
  8c6609ae
06 Dec, 2022 1 commit
- Simplify BOOL_SWITCH macro to fix compiling error on gcc 7 · 8a2ece89
  Tri Dao authored Dec 06, 2022
  
  8a2ece89
29 Nov, 2022 1 commit
- Release training code · 0bf5e500
  Tri Dao authored Nov 28, 2022
  
  0bf5e500
26 Nov, 2022 2 commits
- Fix typo in comments · 9bc63d1e
  Tri Dao authored Nov 25, 2022
  
  9bc63d1e
- Speed up compilation by splitting into separate .cu files · d95ee1a9
  Tri Dao authored Nov 25, 2022
  
  d95ee1a9
17 Nov, 2022 1 commit
- [LayerNorm] Compile for both sm70 and sm80 · 39ed597b
  Tri Dao authored Nov 17, 2022
  
  39ed597b
15 Nov, 2022 2 commits
- Mention that some CUDA extensions have only been tested on A100s · 43ab0b52
  Tri Dao authored Nov 15, 2022
  
  43ab0b52
- [LayerNorm] Check cuda error after querying ctas_per_sm · e4d3013e
  Tri Dao authored Nov 15, 2022
  
  e4d3013e
14 Nov, 2022 2 commits
- Add GPT and ViT models · 2e33fc8e
  Tri Dao authored Nov 13, 2022
  
  2e33fc8e
- Add fused_dense and dropout_add_layernorm CUDA extensions · fa6d1ce4
  Tri Dao authored Nov 13, 2022
  
  fa6d1ce4
13 Nov, 2022 1 commit
- Add fused cross entropy loss · 7c995381
  Tri Dao authored Nov 12, 2022
  
  7c995381
09 Nov, 2022 1 commit
- Fix out-of-bound memory read · 6998e0ec
  Tri Dao authored Nov 09, 2022
  
  6998e0ec
05 Nov, 2022 2 commits
- Parallelize CUDA bwd along seqlen_k instead of seqlen_q · 55778193
  Tri Dao authored Nov 05, 2022
```
This is faster since we only need to do atomic adds on dq, instead of atomic
adds on both dk and dv.
```
  55778193
- Implement rotary embedding in CUDA · ca81f32e
  Tri Dao authored Nov 04, 2022
  
  ca81f32e
25 Oct, 2022 1 commit
- Get rid of o_rows_are_valid since we don't have headdim=16 anymore · c422fee3
  Tri Dao authored Oct 24, 2022
  
  c422fee3