Commits · ae856f3a609a625d83b09b0d6ed2351735ba2947 · gaoqiong / flash-attention

28 Mar, 2024 8 commits
- Remove unnecessary files · ae856f3a
  Woosuk Kwon authored Mar 28, 2024
  
  ae856f3a
- flash_attn -> vllm_flash_attn · 6ac8e63a
  Woosuk Kwon authored Mar 28, 2024
  
  6ac8e63a
- Remove backward pass · 47ad0761
  Woosuk Kwon authored Mar 28, 2024
  
  47ad0761
- Sync up with Dao-AILab/main · 06c23892
  Woosuk Kwon authored Mar 27, 2024
  
  06c23892
- Add the option for the macro and note (#893) · 69dfbf6c
  Driss Guessous authored Mar 27, 2024
  
  69dfbf6c
- Minor fix in compute_attn_1rowblock_splitkv (#900) · 0c07b792
  ljss authored Mar 28, 2024
  
  0c07b792
- Add the option for the macro and note (#893) · 23e8fa5a
  Driss Guessous authored Mar 27, 2024
  
  23e8fa5a
- Minor fix in compute_attn_1rowblock_splitkv (#900) · 3e9414f1
  ljss authored Mar 28, 2024
  
  3e9414f1
26 Mar, 2024 19 commits
- allow smaller page sizes in varlen api · 79681482
  skrider authored Mar 26, 2024
  
  79681482
- add test for page table overflow · a63157ea
  skrider authored Mar 26, 2024
  
  a63157ea
- resolve page offsets absolutely not relatively · 135a1da6
  skrider authored Mar 26, 2024
  
  135a1da6
- revert hardcoded rotcossin thread layout · 802dd6a9
  skrider authored Feb 26, 2024
  
  802dd6a9
- reshape rotary sin/cos copy to align with paged KV copy · 0f5a45ea
  skrider authored Feb 26, 2024
  
  0f5a45ea
- add page size 16 to tests · bc668587
  skrider authored Feb 13, 2024
  
  bc668587
- all working except rotary embedding · bde5aec8
  skrider authored Feb 13, 2024
  
  bde5aec8
- compiles for all h but 128 · fa13c6b0
  skrider authored Feb 13, 2024
  
  fa13c6b0
- tidy flash_fwd_kernel · 3bb71a96
  skrider authored Feb 11, 2024
  
  3bb71a96
- remove print statements · 347a6253
  skrider authored Feb 11, 2024
  
  347a6253
- allow small page sizes in flash api · c05b8570
  skrider authored Feb 11, 2024
  
  c05b8570
- paged copy refactor working for page size 256 · 36916777
  skrider authored Feb 11, 2024
  
  36916777
- tests passing for single page k · 175369fd
  skrider authored Feb 11, 2024
  
  175369fd
- rearrange initial offset computation · 59e76bea
  skrider authored Feb 11, 2024
  
  59e76bea
- implement kv page iteration functions · 70dd0499
  skrider authored Feb 11, 2024
  
  70dd0499
- only test trivial block size · 409431b8
  skrider authored Feb 11, 2024
  
  409431b8
- reshape gmem copy · 14b190bc
  skrider authored Feb 11, 2024
  
  14b190bc
- add print statements for debugging · ac5e78a6
  skrider authored Feb 09, 2024
  
  ac5e78a6
- add print statements for debugging · 8efeb7f5
  skrider authored Feb 08, 2024
  
  8efeb7f5
19 Mar, 2024 1 commit
- [LayerNorm] Update layer_norm_linear · 36587c01
  Tri Dao authored Mar 18, 2024
  
  36587c01
15 Mar, 2024 3 commits
- fix: cast the alibi slopes to torch.float32 (#846) · 6bbc5323
  Markus Krimmel authored Mar 15, 2024
  
  6bbc5323
- Add in, macrosf for defining __grid_constant__ (#852) · 4a73e903
  Driss Guessous authored Mar 15, 2024
  
  4a73e903
- Enable paged attention in varlen forward (#831) · 2a15840f
  Grigory Sizov authored Mar 15, 2024
```
* Enable paged attention in varlen forward

* Format + fix padding
```
  2a15840f
14 Mar, 2024 2 commits
- Support ARM builds (#757) · 26c9e827
  Arvind Sundararajan authored Mar 13, 2024
  
  26c9e827
- Make nvcc threads configurable via environment variable (#885) · 50896ec5
  Chirag Jain authored Mar 14, 2024
  
  50896ec5
02 Mar, 2024 2 commits
- Bump to v2.5.6 · 6c9e60de
  Tri Dao authored Mar 01, 2024
  
  6c9e60de
- [CI] Change torch 2.3.0.dev20240126 to 20240105 for nvcr 24.02 · 6e2fa307
  Tri Dao authored Mar 01, 2024
  
  6e2fa307
21 Feb, 2024 4 commits
- Bump to v2.5.5 · 87a12776
  Tri Dao authored Feb 21, 2024
  
  87a12776
- Enable headdim 256 backward on consumer GPUs (Ampere, Ada) · 2406f288
  Tri Dao authored Feb 21, 2024
  
  2406f288
- Bump to v2.5.4 · 43950dda
  Tri Dao authored Feb 20, 2024
  
  43950dda
- Update Cutlass to v3.4.1 · 4d6b794b
  Tri Dao authored Feb 20, 2024
  
  4d6b794b
20 Feb, 2024 1 commit
- Don't need to reduce row_sum during online softmax · b32efb1a
  Tri Dao authored Feb 20, 2024
  
  b32efb1a