Commits · f1e01c27ba92f62339338113fef0b0cee9e81443 · gaoqiong / flash-attention

15 Jan, 2023 2 commits
- [Gen] Pass qkv_stride to ft_attention kernel for batched generation · f1e01c27
  Tri Dao authored Jan 15, 2023
  
  f1e01c27
- [Gen] Make generation work with Tensor Parallel · 7c219154
  Tri Dao authored Jan 15, 2023
  
  7c219154
13 Jan, 2023 1 commit

[Compilation] Add _NO_HALF2 flags to be consistent with Pytorch · d5098324

Tri Dao authored Jan 12, 2023

https://github.com/pytorch/pytorch/blob/eb7b89771e989f832147b8f28f62c0b43d5d3c12/cmake/Dependencies.cmake#L1693

d5098324

08 Jan, 2023 4 commits
- [Gen] Remove commented code · f95c2fc1
  Tri Dao authored Jan 07, 2023
  
  f95c2fc1
- [Gen] Add timing option · b4859900
  Tri Dao authored Jan 07, 2023
  
  b4859900
- [Gen] Adjust shape of kv_cache when using FT · 0938298e
  Tri Dao authored Jan 07, 2023
  
  0938298e
- [Gen] Implement top-k and top-p sampling · e02fd588
  Tri Dao authored Jan 07, 2023
  
  e02fd588
07 Jan, 2023 6 commits
- [Gen] Test generation with rotary embedding · 11be742a
  Tri Dao authored Jan 07, 2023
  
  11be742a
- Merge pull request #102 from Lamikins/main · 8d9674ed
  Tri Dao authored Jan 07, 2023
```
fixed cross attention typeerror
```
  8d9674ed
- [TP] Implement TensorParallel without sequence parallel · 93383bd5
  Tri Dao authored Jan 07, 2023
  
  93383bd5
- fixed cross attention typeerror · aec35fd6
  Darius Lam authored Jan 07, 2023
  
  aec35fd6
- Bump to v0.2.7 · ce26d3d7
  Tri Dao authored Jan 06, 2023
  
  ce26d3d7
- [LayerNorm] Implement RMS Norm · 6738d947
  Tri Dao authored Jan 06, 2023
  
  6738d947
06 Jan, 2023 1 commit
- [Compilation] Change BOOL_SWITCH to fix Windows compilation · a1f49a2b
  Tri Dao authored Jan 06, 2023
```
Follow xFormers's DISTPATCH_BOOL. Haven't tested it on Windows.
```
  a1f49a2b
04 Jan, 2023 4 commits
- [Gen] Add option to run generation with FT attention kernel · a668890f
  Tri Dao authored Jan 03, 2023
  
  a668890f
- [Gen, FT] Use fp32 accum for FMA · be1afaa2
  Tri Dao authored Jan 03, 2023
  
  be1afaa2
- [Gen, FT] Use tlength instead of params.timestep for rotary · f266fc72
  Tri Dao authored Jan 03, 2023
  
  f266fc72
- [Gen] Add kernel from FasterTransformer for benchmarking · a01d1213
  Tri Dao authored Jan 03, 2023
  
  a01d1213
02 Jan, 2023 2 commits
- [TP] Put parallel embeddings in separate modules · 4cab4de5
  Tri Dao authored Jan 02, 2023
  
  4cab4de5
- [FusedDense] Limit matrix dims to 2M (instead of 64k) · 1ec09ebd
  Tri Dao authored Jan 01, 2023
  
  1ec09ebd
01 Jan, 2023 4 commits
- [Bert] Fix embedding layer norm before embedding dropout · 714c1b4f
  Tri Dao authored Jan 01, 2023
  
  714c1b4f
- [GPT] Refactor function to shard state_dict for TensorParallel · ef1ba918
  Tri Dao authored Jan 01, 2023
  
  ef1ba918
- [FusedDense] Kick off input all_gather before weight dtype conversion · 65b4064b
  Tri Dao authored Dec 31, 2022
  
  65b4064b
- [Loss] Use flash_attn.losses.cross_entropy.CrossEntropyLoss · 71befc19
  Tri Dao authored Dec 31, 2022
  
  71befc19
30 Dec, 2022 4 commits
- [Docker] Set torchmetrics==0.10.3 · cadfa396
  Tri Dao authored Dec 30, 2022
  
  cadfa396
- [Docs] Fix formatting · 43798966
  Tri Dao authored Dec 30, 2022
  
  43798966
- [Docs] Mention that dropout_layer_norm supports all dims up to 6k · 3c7cbfc1
  Tri Dao authored Dec 29, 2022
  
  3c7cbfc1
- [Docs] Mention that XPos's scale_base is recommended to be 512 · 85b8e3d3
  Tri Dao authored Dec 29, 2022
  
  85b8e3d3
29 Dec, 2022 1 commit
- Update training Dockerfile to use flash-attn==0.2.6 · 984d5204
  Tri Dao authored Dec 29, 2022
  
  984d5204
28 Dec, 2022 4 commits
- Merge pull request #95 from Quentin-Anthony/patch-1 · 02961717
  Tri Dao authored Dec 28, 2022
```
Add gpt-neox adoption
```
  02961717
- Add gpt-neox adoption · d2a69a55
  Quentin Anthony authored Dec 28, 2022
  
  d2a69a55
- Bump to v0.2.6 · a6ec1782
  Tri Dao authored Dec 27, 2022
  
  a6ec1782
- Implement generation for GPT · 63670fd8
  Tri Dao authored Dec 27, 2022
  
  63670fd8
27 Dec, 2022 3 commits
- Support loading GPT2 weights from Huggingface · 9d797d88
  Tri Dao authored Dec 27, 2022
  
  9d797d88
- Tweak CrossEntropyLoss to take process_group in init · c6ecd40a
  Tri Dao authored Dec 27, 2022
  
  c6ecd40a
- Implement Tensor Parallel for GPT model · b4018a50
  Tri Dao authored Dec 25, 2022
  
  b4018a50
25 Dec, 2022 3 commits
- Implement Tensor Parallel for GPT2Embeddings · 78225c53
  Tri Dao authored Dec 25, 2022
  
  78225c53
- Implement Tensor Parallel for transformer Block · a8cfe515
  Tri Dao authored Dec 25, 2022
  
  a8cfe515
- Implement TensorParallel for MHA · 1e712ea8
  Tri Dao authored Dec 24, 2022
  
  1e712ea8
24 Dec, 2022 1 commit
- Implement TensorParallel for FusedDense and FusedDenseGeluDense · 226a1b72
  Tri Dao authored Dec 23, 2022
  
  226a1b72