Commits · 3f10cf1db5443b2baa2d2709eb31f8b95b1ee7e1 · OpenDAS / AutoAWQ

23 Dec, 2023 2 commits
- Set default rope_theta on LlamaLikeBlock (#271) · 66adba4a
  Casper authored Dec 23, 2023
  
  66adba4a
- Add Baichuan2 Support (#247) · cef9f113
  Aoyu authored Dec 23, 2023
```
Co-authored-by: Casper <casperbh.96@gmail.com>
```
  cef9f113
22 Dec, 2023 2 commits
- Fused rope theta (#270) · 727172e9
  Casper authored Dec 23, 2023
```
Co-authored-by: Casper Hansen <casperbh96@gmail.com>
```
  727172e9
- Mixtral: Mixture of Experts quantization (#251) · 5b9f3c47
  Casper authored Dec 22, 2023
  
  5b9f3c47
11 Dec, 2023 1 commit
- [`core`] Make AutoAWQ fused modules compatible with HF transformers (#244) · ae24f424
  Younes Belkada authored Dec 11, 2023
  
  ae24f424
15 Nov, 2023 2 commits
- [`core`] Add `is_hf_transformers` flag (#195) · 74d0fe44
  Younes Belkada authored Nov 15, 2023
  
  74d0fe44
- [`core`] Replace `QuantLlamaMLP` with `QuantFusedMLP` (#188) · 3b362c0d
  Younes Belkada authored Nov 15, 2023
  
  3b362c0d
14 Nov, 2023 1 commit
- Fix multi-GPU loading and inference (#190) · 09c73fb2
  Casper authored Nov 14, 2023
  
  09c73fb2
11 Nov, 2023 2 commits
- Adaptive batch sizing (#181) · c5581b27
  Casper authored Nov 11, 2023
  
  c5581b27
- Reset cache on new generation (#178) · df909e83
  Casper authored Nov 11, 2023
  
  df909e83
10 Nov, 2023 1 commit
- New logic for passing past_key_value (#177) · 5db86ec5
  Younes Belkada authored Nov 10, 2023
  
  5db86ec5
04 Nov, 2023 1 commit
- Create fused LlamaLikeModel (#152) · 8110e028
  Casper authored Nov 04, 2023
  
  8110e028
03 Nov, 2023 3 commits
- Fix condition when rolling cache (#150) · 958678df
  Casper authored Nov 04, 2023
  
  958678df
- [`core` / `attention`] Fix fused attention generation with newest transformers version (#146) · 92a403b2
  Younes Belkada authored Nov 04, 2023
```
Co-authored-by: Casper <casperbh.96@gmail.com>
```
  92a403b2
- Fix performance regression (#148) · 3eda6562
  Casper authored Nov 03, 2023
  
  3eda6562
07 Oct, 2023 1 commit
- Only apply attention mask if seqlen is greater than 1 · e94b7f40
  Casper Hansen authored Oct 07, 2023
  
  e94b7f40
06 Oct, 2023 7 commits
- Remove attention sinks · b13e2a85
  Casper Hansen authored Oct 06, 2023
  
  b13e2a85
- Create RoPE module · 69733d2c
  Casper Hansen authored Oct 06, 2023
  
  69733d2c
- Move attention shapes to fused_utils · 306de683
  Casper Hansen authored Oct 06, 2023
  
  306de683
- Create ALiBi module · 428504e4
  Casper Hansen authored Oct 06, 2023
  
  428504e4
- Update comment · 7a3d06d6
  Casper Hansen authored Oct 06, 2023
  
  7a3d06d6
- Use new WindowedCache · e46703d8
  Casper Hansen authored Oct 06, 2023
  
  e46703d8
- Create cache for fused modules · 66b2e233
  Casper Hansen authored Oct 06, 2023
  
  66b2e233
05 Oct, 2023 1 commit
- Remove past_key_value (save 2GB VRAM) · 40e6952a
  Casper Hansen authored Oct 05, 2023
  
  40e6952a
02 Oct, 2023 3 commits
- Mistral fused modules · d5bb4ec8
  Casper Hansen authored Oct 02, 2023
  
  d5bb4ec8
- Fix Falcon n_kv_heads parameter · 80eea43e
  Casper Hansen authored Oct 02, 2023
  
  80eea43e
- Fix unexpected keyword · d016c513
  Casper Hansen authored Oct 02, 2023
  
  d016c513
27 Sep, 2023 5 commits
- Fix apply_rotary_emb() to have both tensors on the same device · 841a2313
  s4rduk4r authored Sep 27, 2023
  
  841a2313
- Clean code · cba9a28c
  Casper Hansen authored Sep 27, 2023
  
  cba9a28c
- Remove unused imports · 707315a6
  Casper Hansen authored Sep 27, 2023
  
  707315a6
- Move cache initialization back to init · 8c80b3e0
  Casper Hansen authored Sep 27, 2023
  
  8c80b3e0
- Roll cache efficiently and correctly · c2cd6535
  Casper Hansen authored Sep 27, 2023
  
  c2cd6535
26 Sep, 2023 2 commits
- Fix edge case · 0392a823
  Casper Hansen authored Sep 26, 2023
  
  0392a823
- Automatically reset/increase cache · 202c6d6a
  Casper Hansen authored Sep 26, 2023
  
  202c6d6a
24 Sep, 2023 1 commit
- Fix typing classes · 4abfefc9
  Vik Paruchuri authored Sep 23, 2023
  
  4abfefc9
21 Sep, 2023 4 commits
- Remove imports · 9060b980
  Casper Hansen authored Sep 21, 2023
  
  9060b980
- Remove fused modules (later) · 37db5759
  Casper Hansen authored Sep 21, 2023
  
  37db5759
- Fix usage of engine · 133dd7a7
  Casper authored Sep 21, 2023
  
  133dd7a7
- Fix module import · fbeea40b
  Casper Hansen authored Sep 21, 2023
  
  fbeea40b
20 Sep, 2023 1 commit
- Add bigcode fused modules (WIP) · b49168f8
  Casper Hansen authored Sep 20, 2023
  
  b49168f8