Commits · 958678df6672fb151aa429eb2d0908fbc49c4e84 · OpenDAS / AutoAWQ

03 Nov, 2023 3 commits
- Fix condition when rolling cache (#150) · 958678df
  Casper authored Nov 04, 2023
  
  958678df
- [`core` / `attention`] Fix fused attention generation with newest transformers version (#146) · 92a403b2
  Younes Belkada authored Nov 04, 2023
```
Co-authored-by: Casper <casperbh.96@gmail.com>
```
  92a403b2
- Fix performance regression (#148) · 3eda6562
  Casper authored Nov 03, 2023
  
  3eda6562
07 Oct, 2023 1 commit
- Only apply attention mask if seqlen is greater than 1 · e94b7f40
  Casper Hansen authored Oct 07, 2023
  
  e94b7f40
06 Oct, 2023 4 commits
- Create RoPE module · 69733d2c
  Casper Hansen authored Oct 06, 2023
  
  69733d2c
- Move attention shapes to fused_utils · 306de683
  Casper Hansen authored Oct 06, 2023
  
  306de683
- Create ALiBi module · 428504e4
  Casper Hansen authored Oct 06, 2023
  
  428504e4
- Use new WindowedCache · e46703d8
  Casper Hansen authored Oct 06, 2023
  
  e46703d8
05 Oct, 2023 1 commit
- Remove past_key_value (save 2GB VRAM) · 40e6952a
  Casper Hansen authored Oct 05, 2023
  
  40e6952a
02 Oct, 2023 1 commit
- Fix unexpected keyword · d016c513
  Casper Hansen authored Oct 02, 2023
  
  d016c513
27 Sep, 2023 5 commits
- Fix apply_rotary_emb() to have both tensors on the same device · 841a2313
  s4rduk4r authored Sep 27, 2023
  
  841a2313
- Clean code · cba9a28c
  Casper Hansen authored Sep 27, 2023
  
  cba9a28c
- Remove unused imports · 707315a6
  Casper Hansen authored Sep 27, 2023
  
  707315a6
- Move cache initialization back to init · 8c80b3e0
  Casper Hansen authored Sep 27, 2023
  
  8c80b3e0
- Roll cache efficiently and correctly · c2cd6535
  Casper Hansen authored Sep 27, 2023
  
  c2cd6535
26 Sep, 2023 2 commits
- Fix edge case · 0392a823
  Casper Hansen authored Sep 26, 2023
  
  0392a823
- Automatically reset/increase cache · 202c6d6a
  Casper Hansen authored Sep 26, 2023
  
  202c6d6a
21 Sep, 2023 1 commit
- Fix usage of engine · 133dd7a7
  Casper authored Sep 21, 2023
  
  133dd7a7
20 Sep, 2023 4 commits
- Update setup · e3b8d9c4
  Casper authored Sep 20, 2023
  
  e3b8d9c4
- Removed unused module · bf76e108
  Casper Hansen authored Sep 20, 2023
  
  bf76e108
- Fix attention shapes. Add repeat interleave. · 2d593b84
  Casper Hansen authored Sep 20, 2023
  
  2d593b84
- Support kv_heads · 3256ffec
  Casper Hansen authored Sep 20, 2023
  
  3256ffec
15 Sep, 2023 2 commits
- fix bug · 8e7059a7
  qwopqwop200 authored Sep 15, 2023
  
  8e7059a7
- suppport windows · e1884728
  qwopqwop200 authored Sep 15, 2023
  
  e1884728
13 Sep, 2023 1 commit
- Set batch size for attention shapes · d3625d1c
  Casper Hansen authored Sep 13, 2023
  
  d3625d1c
12 Sep, 2023 3 commits
- xq view key · bc4f93c2
  Casper Hansen authored Sep 12, 2023
  
  bc4f93c2
- xk_reshape key · 7f8f9f16
  Casper Hansen authored Sep 12, 2023
  
  7f8f9f16
- Refactor view/reshaping into a predefined dict · a8c9afd5
  Casper Hansen authored Sep 12, 2023
  
  a8c9afd5
11 Sep, 2023 2 commits
- Fix attention, support alibi · 7631add1
  Casper Hansen authored Sep 11, 2023
  
  7631add1
- Switch to torch SDPA · d7badefc
  Casper Hansen authored Sep 11, 2023
  
  d7badefc
09 Sep, 2023 3 commits
- Fix alibi, rotary arg, neox, arg · 73c5e2bf
  Casper Hansen authored Sep 09, 2023
  
  73c5e2bf
- Implement ALiBi. · adc5304b
  Casper Hansen authored Sep 09, 2023
  
  adc5304b
- Create torch SDPA implementation · ebbbc3a5
  Casper Hansen authored Sep 09, 2023
  
  ebbbc3a5
08 Sep, 2023 4 commits
- Use apply_rotary_emb · 64e6b3e1
  Casper Hansen authored Sep 08, 2023
  
  64e6b3e1
- Update kernel, remove unused code · a11c313a
  Casper Hansen authored Sep 08, 2023
  
  a11c313a
- Attention works · fbfa9d82
  Casper Hansen authored Sep 08, 2023
  
  fbfa9d82
- Refactor modules, create separate GEMM and GEMV · fe314160
  Casper Hansen authored Sep 08, 2023
  
  fe314160
06 Sep, 2023 3 commits
- Correct input sizes · ab7d68e7
  Casper Hansen authored Sep 06, 2023
  
  ab7d68e7
- Update shapes for cuda kernel · e71181bd
  Casper Hansen authored Sep 06, 2023
  
  e71181bd
- Use hf_rotary per default · b7290868
  Casper Hansen authored Sep 06, 2023
  
  b7290868