Commits · a1576ad1e887c11f4b76f42e9dfaceeb6369cdb8 · gaoqiong / flash-attention

06 Sep, 2023 1 commit
- Bump to v2.2.1 · a1576ad1
  Tri Dao authored Sep 06, 2023
  
  a1576ad1
05 Sep, 2023 1 commit
- Bump to v2.2.0 · 6d673cd9
  Tri Dao authored Sep 05, 2023
  
  6d673cd9
04 Sep, 2023 4 commits
- Set single threaded compilation for CUDA 12.2 so CI doesn't OOM · 4976650f
  Tri Dao authored Sep 03, 2023
  
  4976650f
- Remove constexpr in launch template to fix CI compilation · 6a89b2f1
  Tri Dao authored Sep 03, 2023
  
  6a89b2f1
- Try switching back to Cutlass 3.2.0 · 97ba7a62
  Tri Dao authored Sep 03, 2023
  
  97ba7a62
- Bump to v2.1.2 · 1dc1b6c8
  Tri Dao authored Sep 03, 2023
  
  1dc1b6c8
28 Aug, 2023 1 commit
- Update Cutlass to v3.2.0 · 757058d4
  Tri Dao authored Aug 27, 2023
  
  757058d4
25 Aug, 2023 1 commit
- Change causal mask to be aligned to bottom-right instead of top-left · 9e5e8bc9
  Tri Dao authored Aug 21, 2023
  
  9e5e8bc9
22 Aug, 2023 1 commit
- Bump version to 2.0.9 · 6711b3bc
  Tri Dao authored Aug 22, 2023
  
  6711b3bc
13 Aug, 2023 1 commit
- Bump to v2.0.5 · c5e87b11
  Tri Dao authored Aug 13, 2023
  
  c5e87b11
01 Aug, 2023 2 commits
- Bump to v2.0.4 · d30f2e1c
  Tri Dao authored Aug 01, 2023
  
  d30f2e1c
- Bump to v2.0.3 · a4e5d1ed
  Tri Dao authored Jul 31, 2023
  
  a4e5d1ed
28 Jul, 2023 1 commit

Kirthi Shankar Sivamani authored Jul 28, 2023



* Bump version to 2.0.2
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Update version in Dockerfile
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

---------
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

32a953f4

23 Jul, 2023 1 commit
- Bump to v2.0.1 · b2520724
  Tri Dao authored Jul 23, 2023
  
  b2520724
20 Jul, 2023 1 commit
- remove checkout v2.0.0.post1 from dockerfile · 30fd8c17
  chuanli11 authored Jul 20, 2023
  
  30fd8c17
17 Jul, 2023 2 commits
- FlashAttention-2 release · 4f285b35
  Tri Dao authored Jul 17, 2023
  
  4f285b35
- Bump to v1.0.9 · 6d48e14a
  Tri Dao authored Jul 17, 2023
  
  6d48e14a
03 Jul, 2023 1 commit
- Bump to v1.0.8 · 9610114c
  Tri Dao authored Jul 02, 2023
  
  9610114c
30 May, 2023 1 commit
- Bump version to 1.0.7 · 85b51d61
  Tri Dao authored May 30, 2023
  
  85b51d61
27 May, 2023 1 commit
- bump to v1.0.6 · dd9c3a1f
  Kirthi Shankar Sivamani authored May 26, 2023
```
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
```
  dd9c3a1f
12 May, 2023 1 commit
- Add ninja to pyproject.toml build-system, bump to v1.0.5 · eff9fe6b
  Tri Dao authored May 12, 2023
  
  eff9fe6b
26 Apr, 2023 1 commit
- [Docs] Clearer error message for bwd d > 64, bump to v1.0.4 · ad113948
  Tri Dao authored Apr 26, 2023
  
  ad113948
21 Apr, 2023 2 commits
- Bump version to v1.0.3.post0 · fbbb1078
  Tri Dao authored Apr 21, 2023
  
  fbbb1078
- Bump version to 1.0.3 · 67ef5d28
  Tri Dao authored Apr 21, 2023
  
  67ef5d28
16 Apr, 2023 1 commit
- Bump to v1.0.2 · df1344f8
  Tri Dao authored Apr 15, 2023
  
  df1344f8
12 Apr, 2023 2 commits
- Bump version to v1.0.1, fix Cutlass version · 853ff729
  Tri Dao authored Apr 12, 2023
  
  853ff729
- Bump version to 1.0.0 · 74af0233
  Tri Dao authored Apr 11, 2023
  
  74af0233
29 Mar, 2023 1 commit
- [Training] Fix lightning _PATH import · 009a3e71
  Tri Dao authored Mar 29, 2023
  
  009a3e71
20 Jan, 2023 1 commit
- fix typo in default.yaml · 419ea45b
  Ikko Eltociear Ashimine authored Jan 21, 2023
```
additionaly -> additionally
```
  419ea45b
19 Jan, 2023 1 commit
- Bump to v0.2.8 · 33e0860c
  Tri Dao authored Jan 19, 2023
  
  33e0860c
18 Jan, 2023 1 commit
- [FusedDense] Support relu, rename FusedDenseGeluDense -> FusedMLP · 88173a1a
  Tri Dao authored Jan 17, 2023
  
  88173a1a
07 Jan, 2023 1 commit
- Bump to v0.2.7 · ce26d3d7
  Tri Dao authored Jan 06, 2023
  
  ce26d3d7
01 Jan, 2023 1 commit
- [Loss] Use flash_attn.losses.cross_entropy.CrossEntropyLoss · 71befc19
  Tri Dao authored Dec 31, 2022
  
  71befc19
30 Dec, 2022 3 commits
- [Docker] Set torchmetrics==0.10.3 · cadfa396
  Tri Dao authored Dec 30, 2022
  
  cadfa396
- [Docs] Fix formatting · 43798966
  Tri Dao authored Dec 30, 2022
  
  43798966
- [Docs] Mention that dropout_layer_norm supports all dims up to 6k · 3c7cbfc1
  Tri Dao authored Dec 29, 2022
  
  3c7cbfc1
29 Dec, 2022 1 commit
- Update training Dockerfile to use flash-attn==0.2.6 · 984d5204
  Tri Dao authored Dec 29, 2022
  
  984d5204
27 Dec, 2022 1 commit
- Implement Tensor Parallel for GPT model · b4018a50
  Tri Dao authored Dec 25, 2022
  
  b4018a50
23 Dec, 2022 1 commit
- Add smoothing for CrossEntropyParallel, rename to CrossEntropyLoss · dff68c2b
  Tri Dao authored Dec 23, 2022
  
  dff68c2b
21 Dec, 2022 1 commit
- Fix typo in config: train.gpu -> train.gpu_mem · c2407dec
  Tri Dao authored Dec 21, 2022
  
  c2407dec