Commits · 09c73fb2c5a0e7cd476d9f12875a0645d8b128ad · OpenDAS / AutoAWQ

14 Nov, 2023 1 commit
- Fix multi-GPU loading and inference (#190) · 09c73fb2
  Casper authored Nov 14, 2023
  
  09c73fb2
11 Nov, 2023 4 commits
- Fix cache util logic (#186) · 299c460b
  Casper authored Nov 11, 2023
  
  299c460b
- Pass arguments to AutoConfig (#97) · 7c976752
  Andrey Glushenkov authored Nov 11, 2023
```
Co-authored-by: Casper <casperbh.96@gmail.com>
```
  7c976752
- Adaptive batch sizing (#181) · c5581b27
  Casper authored Nov 11, 2023
  
  c5581b27
- Reset cache on new generation (#178) · df909e83
  Casper authored Nov 11, 2023
  
  df909e83
10 Nov, 2023 2 commits
- New logic for passing past_key_value (#177) · 5db86ec5
  Younes Belkada authored Nov 10, 2023
  
  5db86ec5
- Make sure to delete dummy model (#180) · 1f6f5155
  Casper authored Nov 10, 2023
  
  1f6f5155
06 Nov, 2023 1 commit
- Yi support (#167) · 37c63440
  Casper authored Nov 06, 2023
  
  37c63440
04 Nov, 2023 2 commits
- Create fused LlamaLikeModel (#152) · 8110e028
  Casper authored Nov 04, 2023
  
  8110e028
- Default to safetensors for quantized models (#151) · 84a26861
  Casper authored Nov 04, 2023
  
  84a26861
03 Nov, 2023 3 commits
- Fix condition when rolling cache (#150) · 958678df
  Casper authored Nov 04, 2023
  
  958678df
- [`core` / `attention`] Fix fused attention generation with newest transformers version (#146) · 92a403b2
  Younes Belkada authored Nov 04, 2023
```
Co-authored-by: Casper <casperbh.96@gmail.com>
```
  92a403b2
- Fix performance regression (#148) · 3eda6562
  Casper authored Nov 03, 2023
  
  3eda6562
02 Nov, 2023 2 commits
- Bump to v0.1.6 (#139) · abf44cce
  Casper authored Nov 02, 2023
  
  abf44cce
- Fix init quant (#136) · 89f3feae
  Casper authored Nov 02, 2023
  
  89f3feae
31 Oct, 2023 1 commit
- AwqConfig class (#132) · eea08aa6
  Casper authored Oct 31, 2023
  
  eea08aa6
29 Oct, 2023 1 commit
- Pseudo dequantize function (#127) · f1a2e56e
  Casper authored Oct 29, 2023
  
  f1a2e56e
28 Oct, 2023 4 commits
- Bump to v0.1.5 · b9ed6641
  Casper authored Oct 28, 2023
  
  b9ed6641
- Support Aquila models. (#123) · dc13f0b8
  ldwang authored Oct 28, 2023
```
Signed-off-by: ldwang <ftgreat@gmail.com>
Co-authored-by: ldwang <ftgreat@gmail.com>
Co-authored-by: Casper Hansen <casperbh.96@gmail.com>
```
  dc13f0b8
- Fixing starcoder based models with 15B (#118) · 6c05f669
  Sebastian Bodza authored Oct 28, 2023
  
  6c05f669
- [`core`] Support fp32 / bf16 inference (#121) · 18712d00
  Younes Belkada authored Oct 28, 2023
  
  18712d00
19 Oct, 2023 1 commit
- add comments · bcfdeb38
  twaka authored Oct 17, 2023
  
  bcfdeb38
17 Oct, 2023 1 commit
- gpt_neox · ad45716f
  twaka authored Oct 17, 2023
  
  ad45716f
07 Oct, 2023 1 commit
- Only apply attention mask if seqlen is greater than 1 · e94b7f40
  Casper Hansen authored Oct 07, 2023
  
  e94b7f40
06 Oct, 2023 8 commits
- Bump to v0.1.4 · 0baf5e18
  Casper authored Oct 06, 2023
  
  0baf5e18
- Remove attention sinks · b13e2a85
  Casper Hansen authored Oct 06, 2023
  
  b13e2a85
- Create RoPE module · 69733d2c
  Casper Hansen authored Oct 06, 2023
  
  69733d2c
- Move attention shapes to fused_utils · 306de683
  Casper Hansen authored Oct 06, 2023
  
  306de683
- Create ALiBi module · 428504e4
  Casper Hansen authored Oct 06, 2023
  
  428504e4
- Update comment · 7a3d06d6
  Casper Hansen authored Oct 06, 2023
  
  7a3d06d6
- Use new WindowedCache · e46703d8
  Casper Hansen authored Oct 06, 2023
  
  e46703d8
- Create cache for fused modules · 66b2e233
  Casper Hansen authored Oct 06, 2023
  
  66b2e233
05 Oct, 2023 3 commits
- Remove warning · 661047f1
  Casper authored Oct 05, 2023
  
  661047f1
- Bump to v0.1.3 · a6ecd0db
  Casper authored Oct 05, 2023
  
  a6ecd0db
- Remove past_key_value (save 2GB VRAM) · 40e6952a
  Casper Hansen authored Oct 05, 2023
  
  40e6952a
02 Oct, 2023 4 commits
- Bump to v0.1.2 · eccb8f9c
  Casper Hansen authored Oct 02, 2023
  
  eccb8f9c
- Mistral fused modules · d5bb4ec8
  Casper Hansen authored Oct 02, 2023
  
  d5bb4ec8
- Fix Falcon n_kv_heads parameter · 80eea43e
  Casper Hansen authored Oct 02, 2023
  
  80eea43e
- Fix unexpected keyword · d016c513
  Casper Hansen authored Oct 02, 2023
  
  d016c513
27 Sep, 2023 1 commit
- Bump to v0.1.1 · 6bdcd786
  Casper authored Sep 27, 2023
  
  6bdcd786