Commits · cd089597fdde59d5e413ae055f1845ff8475af4d · gaoqiong / flash-attention

20 Dec, 2023 1 commit
- [LayerNorm] Implement dropout in fused residual + LN/RMSNorm · cd089597
  Tri Dao authored Dec 19, 2023
  
  cd089597
17 Dec, 2023 2 commits
- [CrossEntropy] Test longer sequences · 713bd3aa
  Tri Dao authored Dec 16, 2023
  
  713bd3aa
- [CrossEntropy] Implement logit_scale option · 08124c8f
  Tri Dao authored Dec 16, 2023
  
  08124c8f
01 Dec, 2023 1 commit
- [LayerNorm] Implement layer_norm_linear · 9356a1c0
  Tri Dao authored Nov 30, 2023
  
  9356a1c0
20 Nov, 2023 2 commits
- [CrossEntropy] Simplify the case of large vocab with Tensor Parallel · aaa14741
  Tri Dao authored Nov 19, 2023
  
  aaa14741
- fix flash ce mp large vocab (#673) · abf04a56
  Shijie authored Nov 20, 2023
  
  abf04a56
14 Nov, 2023 1 commit
- [LayerNorm] Add postnorm residual + LayerNorm/RMSNorm in Triton · 01771645
  Tri Dao authored Nov 13, 2023
  
  01771645
13 Nov, 2023 1 commit
- [LayerNorm] Implement residual + LayerNorm/RMSNorm in Triton · 79bd1a2d
  Tri Dao authored Nov 13, 2023
  
  79bd1a2d
03 Oct, 2023 1 commit
- [Gen] Accept cache_batch_idx to index into the KV cache · e279bf8e
  Tri Dao authored Oct 03, 2023
  
  e279bf8e
26 Sep, 2023 1 commit
- Implement local attention · 083e8f52
  Tri Dao authored Sep 24, 2023
```
Co-authored-by: Timothee Lacroix <t@mistral.ai>
```
  083e8f52
24 Sep, 2023 1 commit
- Don't over-allocate dq_accum in case of varlen · 65c234ed
  Tri Dao authored Sep 24, 2023
  
  65c234ed
21 Sep, 2023 2 commits
- Swap seqlen_q and ngroups when seqlen_q=1 (h/t Daniel Haziza) · 2d8ea9a5
  Tri Dao authored Sep 20, 2023
  
  2d8ea9a5
- [Llama] Fix some tests, add tests for Llama 2 and CodeLlama · 0705d271
  Tri Dao authored Sep 20, 2023
  
  0705d271
20 Sep, 2023 3 commits
- [Gen] Simplify decode_speculative · e0fbaa70
  Tri Dao authored Sep 18, 2023
  
  e0fbaa70
- [Gen] Rename max_sequence_len->max_seqlen, sequence_len_offset->seqlen_offset · e6a80264
  Tri Dao authored Sep 18, 2023
  
  e6a80264
- Fix Llama GQA/MQA (#546) · 42832575
  Kevin Hu authored Sep 19, 2023
```
* Fix llama MQA

* Fix permute shape

* Update llama.py
```
  42832575
18 Sep, 2023 2 commits
- [Gen] Don't use ft_attention, use flash_attn_with_kvcache instead · dfe29f5e
  Tri Dao authored Sep 18, 2023
  
  dfe29f5e
- Swap seqlen_q, nheads for MQA when seqlen_q=1 for fwd (h/t Daniel H) · 3250ff3d
  Tri Dao authored Sep 18, 2023
  
  3250ff3d
16 Sep, 2023 2 commits
- Implement rotary embedding in flash_attn_with_kvcache · ccbb14f3
  Tri Dao authored Sep 16, 2023
  
  ccbb14f3
- [CE] Implement CrossEntropyLoss in Triton · 5400fdc4
  Tri Dao authored Sep 15, 2023
  
  5400fdc4
13 Sep, 2023 2 commits
- Simplify the implementation of KVcache attn by appending KV first · 56b7fc6e
  Tri Dao authored Sep 13, 2023
  
  56b7fc6e
- Add tests for Pythia, GPT-JT, and RedPajama models · d0032700
  Tri Dao authored Sep 13, 2023
  
  d0032700
11 Sep, 2023 2 commits
- Add BigCode converters (#532) · 07005806
  Kevin Hu authored Sep 10, 2023
  
  07005806
- [Gen] Fix calling update_graph_cache in tests · 8a733cbd
  Tri Dao authored Sep 10, 2023
  
  8a733cbd
09 Sep, 2023 1 commit
- Inverse state dict for BERT (#527) · 4c91621a
  Kevin Hu authored Sep 09, 2023
  
  4c91621a
07 Sep, 2023 1 commit
- [Gen] Use flash_attn_with_kvcache in generation · a86442f0
  Tri Dao authored Sep 07, 2023
  
  a86442f0
06 Sep, 2023 1 commit
- [Rotary] Set device before launching Triton kernel to avoid error · 97951590
  Tri Dao authored Sep 05, 2023
  
  97951590
05 Sep, 2023 2 commits
- Support cache_seqlens being integer · fd20f16a
  Tri Dao authored Sep 05, 2023
  
  fd20f16a
- [Gen] Refactor decoding function · 913922ca
  Tri Dao authored Sep 04, 2023
  
  913922ca
04 Sep, 2023 4 commits
- Implement flash_attn_with_kvcache · 37c6e054
  Tri Dao authored Sep 04, 2023
  
  37c6e054
- Require CUDA 11.6+, clean up setup.py · 0c04943f
  Tri Dao authored Sep 03, 2023
  
  0c04943f
- Fix test_baichuan · 798858f9
  Tri Dao authored Sep 03, 2023
  
  798858f9
- [Rotary] Implement varlen rotary · b28ec236
  Tri Dao authored Sep 03, 2023
  
  b28ec236
03 Sep, 2023 2 commits
- [Rotary] Speed up rotary kernel when interleaved=True · 1c523c1c
  Tri Dao authored Sep 03, 2023
  
  1c523c1c
- [Rotary] Implement rotary in Triton · 942fcbf0
  Tri Dao authored Sep 03, 2023
  
  942fcbf0
30 Aug, 2023 1 commit
- Support MQA + MP for decoding (#490) · 011ec323
  dan_the_3rd authored Aug 30, 2023
```
Co-authored-by: danthe3rd <danthe3rd>
```
  011ec323
29 Aug, 2023 1 commit
- Implement splitKV attention · b1fbbd83
  Tri Dao authored Aug 29, 2023
  
  b1fbbd83
28 Aug, 2023 1 commit
- [Gen] Clone logits before returning when cg=True · 9f42cb6e
  Tri Dao authored Aug 27, 2023
  
  9f42cb6e
27 Aug, 2023 1 commit
- [GPT] Generalize last_token_only arg to num_last_tokens · f8aea6ea
  Tri Dao authored Aug 26, 2023
  
  f8aea6ea
26 Aug, 2023 1 commit
- [GPT] Test generation when passing in multiple tokens · 371e2065
  Tri Dao authored Aug 26, 2023
  
  371e2065