Commits · 62e981446609156b0c5d2c2a54eb628cce83e66d · gaoqiong / flash-attention

02 Jul, 2023 1 commit
- [Rotary] Make sure frequency calculation is in fp32 · 62e98144
  Tri Dao authored Jul 02, 2023
  
  62e98144
02 Jun, 2023 2 commits
- Merge pull request #255 from beginlner/main · 9818f85f
  Tri Dao authored Jun 02, 2023
```
Fix a bug
```
  9818f85f
- Fix a bug · 8e44c0ee
  ljss authored Jun 02, 2023
  
  8e44c0ee
30 May, 2023 3 commits
- Bump version to 1.0.7 · 85b51d61
  Tri Dao authored May 30, 2023
  
  85b51d61
- [FusedDense] Allocate lt_workspace on input device · 27f8f890
  Tri Dao authored May 30, 2023
  
  27f8f890
- [Gen] Add rotary base as an argument to FT attention kernel · 48bc6eac
  Tri Dao authored May 30, 2023
  
  48bc6eac
27 May, 2023 2 commits
- Merge pull request #243 from ksivaman/bump_version_to_v1_0_6 · 7c766b1b
  Tri Dao authored May 26, 2023
```
bump to v1.0.6
```
  7c766b1b
- bump to v1.0.6 · dd9c3a1f
  Kirthi Shankar Sivamani authored May 26, 2023
```
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
```
  dd9c3a1f
25 May, 2023 5 commits
- Update installation instruction · ce68305c
  Tri Dao authored May 25, 2023
  
  ce68305c
- Merge pull request #241 from ksivaman/fix_compilation_time · cf4f0a39
  Tri Dao authored May 25, 2023
```
Fix compilation time
```
  cf4f0a39
- Re-add ninja · 6d45d0bd
  Kirthi Shankar Sivamani authored May 25, 2023
```
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
```
  6d45d0bd
- Remove torch from pyproject.toml · 852bc40b
  Kirthi Shankar Sivamani authored May 25, 2023
```
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
```
  852bc40b
- Remove ninja from pyproject.toml · c1d117c2
  Kirthi Shankar Sivamani authored May 25, 2023
```
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
```
  c1d117c2
19 May, 2023 3 commits
- Recommend Nvidia's Pytorch container · f0c40b7d
  Tri Dao authored May 19, 2023
  
  f0c40b7d
- Merge pull request #229 from maxhgerlach/local-version · 3cad2ab3
  Tri Dao authored May 19, 2023
```
Allow adding an optional local version to the package version
```
  3cad2ab3
- Allow adding an optional local version to the package version · 31f78a98
  Max H. Gerlach authored May 19, 2023
  
  31f78a98
17 May, 2023 1 commit
- Update roadmap · 40a25c8e
  Tri Dao authored May 17, 2023
  
  40a25c8e
12 May, 2023 2 commits
- Add ninja to pyproject.toml build-system, bump to v1.0.5 · eff9fe6b
  Tri Dao authored May 12, 2023
  
  eff9fe6b
- Merge pull request #193 from anthonyhu/pyproject-build · 36d0a19f
  Tri Dao authored May 11, 2023
```
Use pyproject.toml to specify build dependencies
```
  36d0a19f
06 May, 2023 3 commits
- Merge pull request #202 from fedebotu/main · 5bf7f57d
  Tri Dao authored May 06, 2023
```
[BugFix] avoid bug on ImportError
```
  5bf7f57d
- [BugFix] cannot unpack non-iterable NoneType object · 69f5f7d0
  Federico Berto authored May 07, 2023
  
  69f5f7d0
- [BugFix] cannot unpack non-iterable NoneType object · 3889ba16
  Federico Berto authored May 07, 2023
  
  3889ba16
05 May, 2023 1 commit
- [LLaMa] Fix last norm layer to use RMSNorm instead of LayerNorm · a9a4b4e4
  Tri Dao authored May 04, 2023
  
  a9a4b4e4
27 Apr, 2023 1 commit
- Use pyproject.toml to specify build dependencies · d63cfc35
  Anthony Hu authored Apr 27, 2023
  
  d63cfc35
26 Apr, 2023 1 commit
- [Docs] Clearer error message for bwd d > 64, bump to v1.0.4 · ad113948
  Tri Dao authored Apr 26, 2023
  
  ad113948
21 Apr, 2023 6 commits
- Bump version to v1.0.3.post0 · fbbb1078
  Tri Dao authored Apr 21, 2023
  
  fbbb1078
- Bump version to 1.0.3 · 67ef5d28
  Tri Dao authored Apr 21, 2023
  
  67ef5d28
- [Gen] Minor tweak to allocate_inference_cache · fcab93b4
  Tri Dao authored Apr 21, 2023
  
  fcab93b4
- [Gen] Move allocate_inference_cache to within the model · ba2fe7f3
  Tri Dao authored Apr 20, 2023
  
  ba2fe7f3
- [GPT] Add option to only return the logit for the last token · 3da42d24
  Tri Dao authored Apr 20, 2023
  
  3da42d24
- [Gen] Fix FT kernel smem size, CG when batch size changed · 311d6606
  Tri Dao authored Apr 20, 2023
  
  311d6606
19 Apr, 2023 1 commit
- Implement LLaMa · 96d10f65
  Tri Dao authored Apr 18, 2023
  
  96d10f65
18 Apr, 2023 2 commits
- Implement GatedMlp · b630aef5
  Tri Dao authored Apr 18, 2023
  
  b630aef5
- Have a separate nn.Dropout module in SelfAttention module · ac3b684c
  Tri Dao authored Apr 17, 2023
  
  ac3b684c
16 Apr, 2023 2 commits
- Bump to v1.0.2 · df1344f8
  Tri Dao authored Apr 15, 2023
  
  df1344f8
- Merge pull request #166 from ksivaman/enable_cuda_graph_capture · 635f159e
  Tri Dao authored Apr 16, 2023
```
Enable CUDA graph capture
```
  635f159e
15 Apr, 2023 3 commits
- only 1 thread writes to global mem in fprop · 45567a25
  Kirthi Shankar Sivamani authored Apr 15, 2023
```
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
```
  45567a25
- Merge branch 'HazyResearch:main' into enable_cuda_graph_capture · a0997bc7
  Kirthi Shankar Sivamani authored Apr 14, 2023
  
  a0997bc7
- [Docs] Link to Forbes article · 221a39fd
  Tri Dao authored Apr 14, 2023
  
  221a39fd
14 Apr, 2023 1 commit
- [Gen] Fix FT kernel when using CG · 605655bc
  Tri Dao authored Apr 14, 2023
  
  605655bc