Commits · 6ac8e63a85acdf717783873c67eb4cb648c621a4 · gaoqiong / flash-attention

28 Mar, 2024 1 commit
- flash_attn -> vllm_flash_attn · 6ac8e63a
  Woosuk Kwon authored Mar 28, 2024
  
  6ac8e63a
05 Jan, 2024 1 commit
- [LayerNorm] Switch from CUDA to Triton implementation · abbc1311
  Tri Dao authored Jan 05, 2024
  
  abbc1311
18 Aug, 2023 1 commit
- Run isort and black on python files · f1a73d07
  Tri Dao authored Aug 18, 2023
  
  f1a73d07
23 Jul, 2023 1 commit
- [MLP] Add ParallelMLP · 75e334d4
  Tri Dao authored Jul 22, 2023
  
  75e334d4
22 Jul, 2023 1 commit
- [GPT] Enable FlashAttention for GPT-J · b3177dfa
  Tri Dao authored Jul 21, 2023
  
  b3177dfa
21 Jul, 2023 1 commit
- [Block] Re-enable DropPath · 6fc1e07d
  Tri Dao authored Jul 21, 2023
  
  6fc1e07d
17 Jul, 2023 1 commit
- FlashAttention-2 release · 4f285b35
  Tri Dao authored Jul 17, 2023
  
  4f285b35
02 Jun, 2023 1 commit
- Fix a bug · 8e44c0ee
  ljss authored Jun 02, 2023
  
  8e44c0ee
06 May, 2023 1 commit
- [BugFix] cannot unpack non-iterable NoneType object · 3889ba16
  Federico Berto authored May 07, 2023
  
  3889ba16
21 Apr, 2023 1 commit
- [Gen] Move allocate_inference_cache to within the model · ba2fe7f3
  Tri Dao authored Apr 20, 2023
  
  ba2fe7f3
19 Apr, 2023 1 commit
- Implement LLaMa · 96d10f65
  Tri Dao authored Apr 18, 2023
  
  96d10f65
31 Mar, 2023 1 commit
- [LayerNorm] Implement LN with parallel residual, support dim 8k · 393882bc
  Tri Dao authored Mar 29, 2023
  
  393882bc
22 Mar, 2023 1 commit
- Implement GPT-J · 4d87e4d8
  Tri Dao authored Mar 22, 2023
  
  4d87e4d8
18 Jan, 2023 1 commit
- [FusedDense] Support relu, rename FusedDenseGeluDense -> FusedMLP · 88173a1a
  Tri Dao authored Jan 17, 2023
  
  88173a1a
16 Jan, 2023 3 commits
- [ViT] Support timm checkpoint, add tests · 780e8eea
  Tri Dao authored Jan 16, 2023
  
  780e8eea
- [ViT] Fix extra norm_0, use new LN order in Block · ef085cfc
  Tri Dao authored Jan 15, 2023
  
  ef085cfc
- Reorder LN in Block, support OPT · ff34123b
  Tri Dao authored Jan 15, 2023
  
  ff34123b
07 Jan, 2023 1 commit
- [TP] Implement TensorParallel without sequence parallel · 93383bd5
  Tri Dao authored Jan 07, 2023
  
  93383bd5
25 Dec, 2022 1 commit
- Implement Tensor Parallel for transformer Block · a8cfe515
  Tri Dao authored Dec 25, 2022
  
  a8cfe515
19 Dec, 2022 1 commit
- Implement BERT · 5fb6df0e
  Tri Dao authored Dec 18, 2022
  
  5fb6df0e
14 Nov, 2022 1 commit
- Add MLP, MHA, Block, Embedding modules · d4b320b3
  Tri Dao authored Nov 13, 2022
  
  d4b320b3