Commits · 62e981446609156b0c5d2c2a54eb628cce83e66d · gaoqiong / flash-attention

02 Jul, 2023 1 commit
- [Rotary] Make sure frequency calculation is in fp32 · 62e98144
  Tri Dao authored Jul 02, 2023
  
  62e98144
30 May, 2023 1 commit
- [Gen] Add rotary base as an argument to FT attention kernel · 48bc6eac
  Tri Dao authored May 30, 2023
  
  48bc6eac
14 Mar, 2023 1 commit
- [Rotary] Implement GPT-J style (interleaved) rotary · e45a46a5
  Tri Dao authored Mar 14, 2023
  
  e45a46a5
30 Dec, 2022 1 commit
- [Docs] Mention that XPos's scale_base is recommended to be 512 · 85b8e3d3
  Tri Dao authored Dec 29, 2022
  
  85b8e3d3
25 Dec, 2022 1 commit
- Implement TensorParallel for MHA · 1e712ea8
  Tri Dao authored Dec 24, 2022
  
  1e712ea8
23 Dec, 2022 1 commit
- Simplify FusedDense · e68ebbe8
  Tri Dao authored Dec 22, 2022
  
  e68ebbe8
21 Dec, 2022 1 commit
- Implement XPos (Sun et al.) · 496e4f52
  Tri Dao authored Dec 21, 2022
  
  496e4f52
17 Dec, 2022 1 commit
- add asserts for sin shape · ee8984d2
  Alexander Ploshkin authored Dec 17, 2022
  
  ee8984d2
16 Dec, 2022 1 commit
- fix slicing dimensions · c7c66976
  Alexander Ploshkin authored Dec 16, 2022
  
  c7c66976
15 Dec, 2022 1 commit
- Remove redundant shape asserts in rotary embeddings · 96656b93
  Alexander Ploshkin authored Dec 15, 2022
  
  96656b93
18 Nov, 2022 2 commits
- Add PatchEmbed · 0fa5c0d7
  Tri Dao authored Nov 17, 2022
  
  0fa5c0d7
- Add __init__.py files to subdirectories for installation · ece539ab
  Tri Dao authored Nov 17, 2022
  
  ece539ab
17 Nov, 2022 1 commit
- [Rotary] Customize base, support seqlen_offset · 71f674ae
  Tri Dao authored Nov 17, 2022
  
  71f674ae
14 Nov, 2022 1 commit
- Add MLP, MHA, Block, Embedding modules · d4b320b3
  Tri Dao authored Nov 13, 2022
  
  d4b320b3