Commits · 3250ff3d829afceea80ecf7299bef629c93a6b47 · gaoqiong / flash-attention

07 Sep, 2023 1 commit
- [Gen] Use flash_attn_with_kvcache in generation · a86442f0
  Tri Dao authored Sep 07, 2023
  
  a86442f0
05 Sep, 2023 1 commit
- [Gen] Refactor decoding function · 913922ca
  Tri Dao authored Sep 04, 2023
  
  913922ca
04 Sep, 2023 1 commit
- Implement flash_attn_with_kvcache · 37c6e054
  Tri Dao authored Sep 04, 2023
  
  37c6e054
29 Aug, 2023 1 commit
- [Gen] Minor fix to modify logits for top_p · 8a326bbc
  Tri Dao authored Aug 29, 2023
  
  8a326bbc
28 Aug, 2023 1 commit
- [Gen] Clone logits before returning when cg=True · 9f42cb6e
  Tri Dao authored Aug 27, 2023
  
  9f42cb6e
27 Aug, 2023 1 commit
- [GPT] Generalize last_token_only arg to num_last_tokens · f8aea6ea
  Tri Dao authored Aug 26, 2023
  
  f8aea6ea
26 Aug, 2023 2 commits
- [Gen] Fix decode function not using top_p during iterative decoding · 7a3bd55f
  Tri Dao authored Aug 26, 2023
  
  7a3bd55f
- [Gen] Refactor decode function a bit · 847abe65
  Tri Dao authored Aug 26, 2023
  
  847abe65
18 Aug, 2023 1 commit
- Run isort and black on python files · f1a73d07
  Tri Dao authored Aug 18, 2023
  
  f1a73d07
21 Apr, 2023 4 commits
- [Gen] Minor tweak to allocate_inference_cache · fcab93b4
  Tri Dao authored Apr 21, 2023
  
  fcab93b4
- [Gen] Move allocate_inference_cache to within the model · ba2fe7f3
  Tri Dao authored Apr 20, 2023
  
  ba2fe7f3
- [GPT] Add option to only return the logit for the last token · 3da42d24
  Tri Dao authored Apr 20, 2023
  
  3da42d24
- [Gen] Fix FT kernel smem size, CG when batch size changed · 311d6606
  Tri Dao authored Apr 20, 2023
  
  311d6606
14 Apr, 2023 1 commit
- [Gen] Fix FT kernel when using CG · 605655bc
  Tri Dao authored Apr 14, 2023
  
  605655bc
13 Apr, 2023 1 commit
- [Gen] Measure prompt processing + decoding time, not just decoding · 1c9ef9b3
  Tri Dao authored Apr 13, 2023
  
  1c9ef9b3
29 Mar, 2023 1 commit
- [FT] Fix FT's single query attention for bf16 hdim128 rotary · f5d0fbd4
  Tri Dao authored Mar 28, 2023
  
  f5d0fbd4
22 Mar, 2023 1 commit
- Implement GPT-J · 4d87e4d8
  Tri Dao authored Mar 22, 2023
  
  4d87e4d8
23 Jan, 2023 1 commit
- [OPT] Load fp16 weights on CPU before moving to GPU · 78b7a1dc
  Tri Dao authored Jan 22, 2023
  
  78b7a1dc
18 Jan, 2023 1 commit
- [Gen] Add OPT to generation test · f68d41ec
  Tri Dao authored Jan 17, 2023
  
  f68d41ec
15 Jan, 2023 1 commit
- [Gen] Make generation work with Tensor Parallel · 7c219154
  Tri Dao authored Jan 15, 2023
  
  7c219154
08 Jan, 2023 3 commits
- [Gen] Remove commented code · f95c2fc1
  Tri Dao authored Jan 07, 2023
  
  f95c2fc1
- [Gen] Add timing option · b4859900
  Tri Dao authored Jan 07, 2023
  
  b4859900
- [Gen] Implement top-k and top-p sampling · e02fd588
  Tri Dao authored Jan 07, 2023
  
  e02fd588
04 Jan, 2023 1 commit
- [Gen] Add option to run generation with FT attention kernel · a668890f
  Tri Dao authored Jan 03, 2023
  
  a668890f
28 Dec, 2022 2 commits
- Bump to v0.2.6 · a6ec1782
  Tri Dao authored Dec 27, 2022
  
  a6ec1782
- Implement generation for GPT · 63670fd8
  Tri Dao authored Dec 27, 2022
  
  63670fd8