Commits · a157cc8c9bf4460da70243bc100a363b5daad9ba · gaoqiong / flash-attention

23 Jul, 2023 1 commit
- [FT] Implement MQA/GQA · a157cc8c
  Tri Dao authored Jul 22, 2023
  
  a157cc8c
06 Jul, 2023 1 commit
- [FT] rotary_cos/sin should have batch_size dimension · 2800efc7
  Tri Dao authored Jul 06, 2023
  
  2800efc7
03 Jul, 2023 1 commit
- [FT] rotary_cos/sin should have shape (dim) instead of (seqlen, dim) · 3a9bfd07
  Tri Dao authored Jul 03, 2023
  
  3a9bfd07
02 Jul, 2023 1 commit
- [Rotary] Make sure frequency calculation is in fp32 · 62e98144
  Tri Dao authored Jul 02, 2023
  
  62e98144
30 May, 2023 1 commit
- [Gen] Add rotary base as an argument to FT attention kernel · 48bc6eac
  Tri Dao authored May 30, 2023
  
  48bc6eac
21 Apr, 2023 1 commit
- [Gen] Fix FT kernel smem size, CG when batch size changed · 311d6606
  Tri Dao authored Apr 20, 2023
  
  311d6606
29 Mar, 2023 1 commit
- [FT] Fix FT's single query attention for bf16 hdim128 rotary · f5d0fbd4
  Tri Dao authored Mar 28, 2023
  
  f5d0fbd4
15 Mar, 2023 1 commit
- Support H100 for other CUDA extensions · dc08ea1c
  Tri Dao authored Mar 15, 2023
  
  dc08ea1c
15 Jan, 2023 2 commits
- [Gen] Pass qkv_stride to ft_attention kernel for batched generation · f1e01c27
  Tri Dao authored Jan 15, 2023
  
  f1e01c27
- [Gen] Make generation work with Tensor Parallel · 7c219154
  Tri Dao authored Jan 15, 2023
  
  7c219154
04 Jan, 2023 3 commits
- [Gen, FT] Use fp32 accum for FMA · be1afaa2
  Tri Dao authored Jan 03, 2023
  
  be1afaa2
- [Gen, FT] Use tlength instead of params.timestep for rotary · f266fc72
  Tri Dao authored Jan 03, 2023
  
  f266fc72
- [Gen] Add kernel from FasterTransformer for benchmarking · a01d1213
  Tri Dao authored Jan 03, 2023
  
  a01d1213