Commits · a1576ad1e887c11f4b76f42e9dfaceeb6369cdb8 · gaoqiong / flash-attention

06 Sep, 2023 2 commits
- Bump to v2.2.1 · a1576ad1
  Tri Dao authored Sep 06, 2023
  
  a1576ad1
- [Rotary] Set device before launching Triton kernel to avoid error · 97951590
  Tri Dao authored Sep 05, 2023
  
  97951590
05 Sep, 2023 4 commits
- Bump to v2.2.0 · 6d673cd9
  Tri Dao authored Sep 05, 2023
  
  6d673cd9
- Create __init__.py for ops/triton dir (#516) · 8e893f09
  Kyeongpil Kang authored Sep 06, 2023
  
  8e893f09
- Support cache_seqlens being integer · fd20f16a
  Tri Dao authored Sep 05, 2023
  
  fd20f16a
- [Gen] Refactor decoding function · 913922ca
  Tri Dao authored Sep 04, 2023
  
  913922ca
04 Sep, 2023 9 commits
- [MLP] Implement SwiGLU with torch jiterator · 3557e0bb
  Tri Dao authored Sep 04, 2023
  
  3557e0bb
- Implement flash_attn_with_kvcache · 37c6e054
  Tri Dao authored Sep 04, 2023
  
  37c6e054
- Set single threaded compilation for CUDA 12.2 so CI doesn't OOM · 4976650f
  Tri Dao authored Sep 03, 2023
  
  4976650f
- Remove constexpr in launch template to fix CI compilation · 6a89b2f1
  Tri Dao authored Sep 03, 2023
  
  6a89b2f1
- Try switching back to Cutlass 3.2.0 · 97ba7a62
  Tri Dao authored Sep 03, 2023
  
  97ba7a62
- Bump to v2.1.2 · 1dc1b6c8
  Tri Dao authored Sep 03, 2023
  
  1dc1b6c8
- Fix test_baichuan · 798858f9
  Tri Dao authored Sep 03, 2023
  
  798858f9
- [Gen] Add back num_last_tokens in gpt.py · 7b33743a
  Tri Dao authored Sep 03, 2023
  
  7b33743a
- [Rotary] Implement varlen rotary · b28ec236
  Tri Dao authored Sep 03, 2023
  
  b28ec236
03 Sep, 2023 4 commits
- [Rotary] Clean up rotary Triton implementation a bit · 861c8257
  Tri Dao authored Sep 03, 2023
  
  861c8257
- [Rotary] Speed up rotary kernel when interleaved=True · 1c523c1c
  Tri Dao authored Sep 03, 2023
  
  1c523c1c
- [Rotary] Pass max_seqlen from mha.py to rotary during inference · de2949f3
  Tri Dao authored Sep 03, 2023
  
  de2949f3
- [Rotary] Implement rotary in Triton · 942fcbf0
  Tri Dao authored Sep 03, 2023
  
  942fcbf0
30 Aug, 2023 3 commits
- Support LLaMa2 and CodeLLaMa (#491) · c9d4a816
  dan_the_3rd authored Aug 30, 2023
```
Co-authored-by: danthe3rd <danthe3rd>
```
  c9d4a816
- Support MQA + MP for decoding (#490) · 011ec323
  dan_the_3rd authored Aug 30, 2023
```
Co-authored-by: danthe3rd <danthe3rd>
```
  011ec323
- [bugfix] handle_x not define when using checkpoint_lvl = 2 (#502) · 0cb595ad
  GAOXinyu authored Aug 30, 2023
```
when using checkpoint_lvl=2, we all_gather_raw(x) without async_op=True.
So we don't need to wait for handle. Just skip.
```
  0cb595ad
29 Aug, 2023 2 commits
- [Gen] Minor fix to modify logits for top_p · 8a326bbc
  Tri Dao authored Aug 29, 2023
  
  8a326bbc
- add unpad_input_for_concatenated_sequences (#499) · 8f6f48d8
  Su Zhu authored Aug 29, 2023
```
* add unpad_input_for_concatenated_sequences

* modify docstring
```
  8f6f48d8
28 Aug, 2023 2 commits
- Update Cutlass to v3.2.0 · 757058d4
  Tri Dao authored Aug 27, 2023
  
  757058d4
- [Gen] Clone logits before returning when cg=True · 9f42cb6e
  Tri Dao authored Aug 27, 2023
  
  9f42cb6e
27 Aug, 2023 1 commit
- [GPT] Generalize last_token_only arg to num_last_tokens · f8aea6ea
  Tri Dao authored Aug 26, 2023
  
  f8aea6ea
26 Aug, 2023 3 commits
- [Gen] Fix decode function not using top_p during iterative decoding · 7a3bd55f
  Tri Dao authored Aug 26, 2023
  
  7a3bd55f
- [Gen] Refactor decode function a bit · 847abe65
  Tri Dao authored Aug 26, 2023
  
  847abe65
- Change causal for CrossAttention in mha.py to align to bottom right · a2974e85
  Tri Dao authored Aug 26, 2023
  
  a2974e85
25 Aug, 2023 2 commits
- Move pyproject.toml to flash-attn and tests dir to avoid PEP 517 · 73bd3f3b
  Tri Dao authored Aug 25, 2023
  
  73bd3f3b
- Change causal mask to be aligned to bottom-right instead of top-left · 9e5e8bc9
  Tri Dao authored Aug 21, 2023
  
  9e5e8bc9
24 Aug, 2023 1 commit
- add llama support to GPTPreTrainedModel.from_pretrained (#479) · e0b09891
  Aman Gupta Karmani authored Aug 24, 2023
  
  e0b09891
22 Aug, 2023 2 commits
- Bump version to 2.0.9 · 6711b3bc
  Tri Dao authored Aug 22, 2023
  
  6711b3bc
- [GPT] Fix loading weights from HF hub · ef6d8c75
  Tri Dao authored Aug 21, 2023
  
  ef6d8c75
21 Aug, 2023 1 commit
- FEAT: add codes which supporting for baichuan-inc/Baichuan-7B (#425) · a8c35b4f
  GAOXinyu authored Aug 22, 2023
  
  a8c35b4f
20 Aug, 2023 2 commits
- handle uneven heads across ranks when combining state_dicts; resolves #467 (#468) · 25d6b1db
  Xuechen Li authored Aug 20, 2023
```
* q

* add comment.
```
  25d6b1db
- Import torch before flash_attn_2_cuda · d431f167
  Tri Dao authored Aug 19, 2023
  
  d431f167
19 Aug, 2023 1 commit

map custom model state_dict back to huggingface format (#465) · 7fcd3e6a

Xuechen Li authored Aug 18, 2023

* fix name.

* set inv function.

* add map back function.

* handle gqa.

* add type annotation to avoid confusion.

* fix docstr.

* test inverse remap logic.

7fcd3e6a

18 Aug, 2023 1 commit
- Run isort and black on python files · f1a73d07
  Tri Dao authored Aug 18, 2023
  
  f1a73d07