Commits · 3a9bfd076f98746c73362328958dbc68d145fbec · gaoqiong / flash-attention

03 Jul, 2023 1 commit
- [FT] rotary_cos/sin should have shape (dim) instead of (seqlen, dim) · 3a9bfd07
  Tri Dao authored Jul 03, 2023
  
  3a9bfd07
02 Jul, 2023 1 commit
- [Rotary] Make sure frequency calculation is in fp32 · 62e98144
  Tri Dao authored Jul 02, 2023
  
  62e98144
30 May, 2023 1 commit
- [Gen] Add rotary base as an argument to FT attention kernel · 48bc6eac
  Tri Dao authored May 30, 2023
  
  48bc6eac
29 Mar, 2023 1 commit
- [FT] Fix FT's single query attention for bf16 hdim128 rotary · f5d0fbd4
  Tri Dao authored Mar 28, 2023
  
  f5d0fbd4
04 Jan, 2023 1 commit
- [Gen] Add kernel from FasterTransformer for benchmarking · a01d1213
  Tri Dao authored Jan 03, 2023
  
  a01d1213