Commits · c3b219665292c61a51153d0ded4473c494296382 · gaoqiong / flash-attention

22 Dec, 2023 1 commit
- Add Alibi to MHA, test with Baichuan-13B · c3b21966
  Tri Dao authored Dec 21, 2023
  
  c3b21966
21 Sep, 2023 1 commit
- [Llama] Fix some tests, add tests for Llama 2 and CodeLlama · 0705d271
  Tri Dao authored Sep 20, 2023
  
  0705d271
20 Sep, 2023 3 commits
- [Gen] Simplify decode_speculative · e0fbaa70
  Tri Dao authored Sep 18, 2023
  
  e0fbaa70
- [Gen] Rename max_sequence_len->max_seqlen, sequence_len_offset->seqlen_offset · e6a80264
  Tri Dao authored Sep 18, 2023
  
  e6a80264
- Fix Llama GQA/MQA (#546) · 42832575
  Kevin Hu authored Sep 19, 2023
```
* Fix llama MQA

* Fix permute shape

* Update llama.py
```
  42832575
18 Sep, 2023 1 commit
- [Gen] Don't use ft_attention, use flash_attn_with_kvcache instead · dfe29f5e
  Tri Dao authored Sep 18, 2023
  
  dfe29f5e
16 Sep, 2023 1 commit
- Implement rotary embedding in flash_attn_with_kvcache · ccbb14f3
  Tri Dao authored Sep 16, 2023
  
  ccbb14f3
13 Sep, 2023 1 commit
- Add tests for Pythia, GPT-JT, and RedPajama models · d0032700
  Tri Dao authored Sep 13, 2023
  
  d0032700
11 Sep, 2023 2 commits
- Add BigCode converters (#532) · 07005806
  Kevin Hu authored Sep 10, 2023
  
  07005806
- [Gen] Fix calling update_graph_cache in tests · 8a733cbd
  Tri Dao authored Sep 10, 2023
  
  8a733cbd
09 Sep, 2023 1 commit
- Inverse state dict for BERT (#527) · 4c91621a
  Kevin Hu authored Sep 09, 2023
  
  4c91621a
07 Sep, 2023 1 commit
- [Gen] Use flash_attn_with_kvcache in generation · a86442f0
  Tri Dao authored Sep 07, 2023
  
  a86442f0
06 Sep, 2023 1 commit
- [Rotary] Set device before launching Triton kernel to avoid error · 97951590
  Tri Dao authored Sep 05, 2023
  
  97951590
05 Sep, 2023 2 commits
- Support cache_seqlens being integer · fd20f16a
  Tri Dao authored Sep 05, 2023
  
  fd20f16a
- [Gen] Refactor decoding function · 913922ca
  Tri Dao authored Sep 04, 2023
  
  913922ca
04 Sep, 2023 1 commit
- Fix test_baichuan · 798858f9
  Tri Dao authored Sep 03, 2023
  
  798858f9
03 Sep, 2023 1 commit
- [Rotary] Implement rotary in Triton · 942fcbf0
  Tri Dao authored Sep 03, 2023
  
  942fcbf0
30 Aug, 2023 1 commit
- Support MQA + MP for decoding (#490) · 011ec323
  dan_the_3rd authored Aug 30, 2023
```
Co-authored-by: danthe3rd <danthe3rd>
```
  011ec323
28 Aug, 2023 1 commit
- [Gen] Clone logits before returning when cg=True · 9f42cb6e
  Tri Dao authored Aug 27, 2023
  
  9f42cb6e
27 Aug, 2023 1 commit
- [GPT] Generalize last_token_only arg to num_last_tokens · f8aea6ea
  Tri Dao authored Aug 26, 2023
  
  f8aea6ea
26 Aug, 2023 3 commits
- [GPT] Test generation when passing in multiple tokens · 371e2065
  Tri Dao authored Aug 26, 2023
  
  371e2065
- [GPT] Move more tests to test_gpt.py · c000c3a2
  Tri Dao authored Aug 26, 2023
  
  c000c3a2
- [GPT] Move GPT and OPT generation tests to test_{gpt,opt}.py · 9b713872
  Tri Dao authored Aug 26, 2023
  
  9b713872
22 Aug, 2023 1 commit
- [GPT] Fix loading weights from HF hub · ef6d8c75
  Tri Dao authored Aug 21, 2023
  
  ef6d8c75
21 Aug, 2023 1 commit
- FEAT: add codes which supporting for baichuan-inc/Baichuan-7B (#425) · a8c35b4f
  GAOXinyu authored Aug 22, 2023
  
  a8c35b4f
19 Aug, 2023 2 commits
- Run isort and black on test files · 0e8c46ae
  Tri Dao authored Aug 18, 2023
  
  0e8c46ae
- map custom model state_dict back to huggingface format (#465) · 7fcd3e6a
  Xuechen Li authored Aug 18, 2023
```
* fix name.

* set inv function.

* add map back function.

* handle gqa.

* add type annotation to avoid confusion.

* fix docstr.

* test inverse remap logic.
```
  7fcd3e6a
18 Aug, 2023 2 commits

support when num_heads is not divisible by world_size; resolves #459 (#461) · bb4cded1

Xuechen Li authored Aug 18, 2023

* uneql rank.

* trim.

* enable passing in number of heads for each rank.

* simplify.

* simplify.

* cleanup.

* fix col parallel.

* fix bug with row parallel.

* fit out proj.

* refac.

* fix sharding logic.

* refac sharding.

* refac.

* support multiple of.

* make fn reuseable.

* fix bug in dimensions.

* scaffold.

* test uneven heads.

* fix test by adding barrier.

* refac.

* reuse code.

* clean up.

bb4cded1

[ViT] Minor fix so it runs · a81900d4
Tri Dao authored Aug 17, 2023

a81900d4

15 Aug, 2023 1 commit

enable loading hf llama checkpoints for training (#446) · 0f7853c6

Xuechen Li authored Aug 15, 2023

* prelim.

* add hf convertion fn.

* mlp.

* change name.

* fix bug.

* inverse permute.

* change comment.

* revert style changes.

* fix.

* add doc.

* revert.

* enable load safe.

* fix safe load.

* fix import.

* fix typing-related lints.

* fix ckpt loading logic.

* make single gpu work.

* test with parallel.

* ckpt format.

* enable pretrained state dict.

* remove unused imports.

* remove unused.

* mark idea related.

0f7853c6

29 Jul, 2023 1 commit
- [GPT] Implement parallel LLaMa · 184b992d
  Tri Dao authored Jul 28, 2023
  
  184b992d
26 Jul, 2023 2 commits
- [GPT] Add LLaMa-13B to test · 56ccaff1
  Tri Dao authored Jul 26, 2023
  
  56ccaff1
- [Rotary] Fix tests when loading state dict with rotary inv_freqs · 8e9820a5
  Tri Dao authored Jul 26, 2023
  
  8e9820a5
23 Jul, 2023 2 commits
- [GPT] Implement Falcon · d38357dd
  Tri Dao authored Jul 23, 2023
  
  d38357dd
- [MHA] Implement MQA/GQA · 425dbcb6
  Tri Dao authored Jul 23, 2023
  
  425dbcb6
22 Jul, 2023 1 commit
- [GPT] Enable FlashAttention for GPT-J · b3177dfa
  Tri Dao authored Jul 21, 2023
  
  b3177dfa
02 Jul, 2023 1 commit
- [Rotary] Make sure frequency calculation is in fp32 · 62e98144
  Tri Dao authored Jul 02, 2023
  
  62e98144
30 May, 2023 1 commit
- [Gen] Add rotary base as an argument to FT attention kernel · 48bc6eac
  Tri Dao authored May 30, 2023
  
  48bc6eac
05 May, 2023 1 commit
- [LLaMa] Fix last norm layer to use RMSNorm instead of LayerNorm · a9a4b4e4
  Tri Dao authored May 04, 2023
  
  a9a4b4e4
21 Apr, 2023 1 commit
- [Gen] Fix FT kernel smem size, CG when batch size changed · 311d6606
  Tri Dao authored Apr 20, 2023
  
  311d6606