Commits · dfe29f5e2bb65ab4920cdfb45eb9d4a256401ab8 · gaoqiong / flash-attention

18 Sep, 2023 1 commit
- [Gen] Don't use ft_attention, use flash_attn_with_kvcache instead · dfe29f5e
  Tri Dao authored Sep 18, 2023
  
  dfe29f5e
13 Sep, 2023 1 commit
- Add tests for Pythia, GPT-JT, and RedPajama models · d0032700
  Tri Dao authored Sep 13, 2023
  
  d0032700
11 Sep, 2023 1 commit
- [Gen] Fix calling update_graph_cache in tests · 8a733cbd
  Tri Dao authored Sep 10, 2023
  
  8a733cbd
05 Sep, 2023 1 commit
- [Gen] Refactor decoding function · 913922ca
  Tri Dao authored Sep 04, 2023
  
  913922ca
19 Aug, 2023 1 commit
- Run isort and black on test files · 0e8c46ae
  Tri Dao authored Aug 18, 2023
  
  0e8c46ae
26 Jul, 2023 1 commit
- [Rotary] Fix tests when loading state dict with rotary inv_freqs · 8e9820a5
  Tri Dao authored Jul 26, 2023
  
  8e9820a5
23 Jul, 2023 1 commit
- [MHA] Implement MQA/GQA · 425dbcb6
  Tri Dao authored Jul 23, 2023
  
  425dbcb6
22 Jul, 2023 1 commit
- [GPT] Enable FlashAttention for GPT-J · b3177dfa
  Tri Dao authored Jul 21, 2023
  
  b3177dfa
19 Apr, 2023 1 commit
- Implement LLaMa · 96d10f65
  Tri Dao authored Apr 18, 2023
  
  96d10f65
14 Apr, 2023 1 commit
- [Gen] Fix FT kernel when using CG · 605655bc
  Tri Dao authored Apr 14, 2023
  
  605655bc
31 Mar, 2023 1 commit
- [LayerNorm] Implement LN with parallel residual, support dim 8k · 393882bc
  Tri Dao authored Mar 29, 2023
  
  393882bc
29 Mar, 2023 1 commit
- Implement GPT-NeoX · 993d1244
  Tri Dao authored Mar 29, 2023
  
  993d1244
22 Mar, 2023 1 commit
- Implement GPT-J · 4d87e4d8
  Tri Dao authored Mar 22, 2023
  
  4d87e4d8