Commits · 34085edc92c30be141fa10defc0e3eb43eaa9d4f · OpenDAS / AutoAWQ

03 Feb, 2024 2 commits
- Marlin symmetric quantization and inference (#320) · 34085edc
  Ilyas Moutawwakil authored Feb 03, 2024
```
Co-authored-by: Casper <casperbh.96@gmail.com>
```
  34085edc
- AMD ROCM Support (#315) · f018d2b7
  Ilyas Moutawwakil authored Feb 03, 2024
  
  f018d2b7
26 Jan, 2024 1 commit
- Fixes a breaking change in autoawq (#325) · 8c78db4d
  Younes Belkada authored Jan 26, 2024
  
  8c78db4d
24 Jan, 2024 3 commits
- Evaluation: Add more evals (#283) · 7133575a
  Casper authored Jan 24, 2024
  
  7133575a
- Up to 60% faster context processing (#316) · 149236e4
  Casper authored Jan 24, 2024
  
  149236e4
- Torch only inference + any-device quantization (#319) · c6c7b065
  Casper authored Jan 24, 2024
  
  c6c7b065
21 Jan, 2024 1 commit
- Exllama kernels support (#313) · 2fcbf268
  Ilyas Moutawwakil authored Jan 21, 2024
```
Co-authored-by: Casper <casperbh.96@gmail.com>
```
  2fcbf268
07 Jan, 2024 1 commit
- GGUF compatible quantization (2, 3, 4 bit / any bit) (#285) · a3db8099
  Casper authored Jan 07, 2024
  
  a3db8099
04 Jan, 2024 1 commit
- Add CPU-loaded multi-GPU quantization (#289) · 46415f5a
  Blake Wyatt authored Jan 04, 2024
  
  46415f5a
28 Dec, 2023 1 commit
- AWQ: Separate the AWQ kernels to separate repository (#279) · 2edb3f6f
  Casper authored Dec 28, 2023
```
Co-authored-by: Casper Hansen <casperbh96@gmail.com>
```
  2edb3f6f
23 Dec, 2023 4 commits
- Bump to v0.1.8 (#274) · 3f10cf1d
  Casper authored Dec 23, 2023
  
  3f10cf1d
- Set default rope_theta on LlamaLikeBlock (#271) · 66adba4a
  Casper authored Dec 23, 2023
  
  66adba4a
- Add Baichuan2 Support (#247) · cef9f113
  Aoyu authored Dec 23, 2023
```
Co-authored-by: Casper <casperbh.96@gmail.com>
```
  cef9f113
- FEAT: add llava to autoawq (#250) · 9e8e28b2
  Younes Belkada authored Dec 23, 2023
```
Co-authored-by: Casper <casperbh.96@gmail.com>
```
  9e8e28b2
22 Dec, 2023 2 commits
- Fused rope theta (#270) · 727172e9
  Casper authored Dec 23, 2023
```
Co-authored-by: Casper Hansen <casperbh96@gmail.com>
```
  727172e9
- Mixtral: Mixture of Experts quantization (#251) · 5b9f3c47
  Casper authored Dec 22, 2023
  
  5b9f3c47
16 Dec, 2023 1 commit
- Fix quantization issue with transformers >= 4.36.0 (#264) · 2350a4d0
  Younes Belkada authored Dec 16, 2023
  
  2350a4d0
11 Dec, 2023 3 commits
- FEAT: Add possibility of skipping modules when quantizing (#248) · 9c3dfa07
  Younes Belkada authored Dec 11, 2023
  
  9c3dfa07
- [`core`] Fix quantization issues with transformers==4.36.0 (#249) · 78b59d73
  Younes Belkada authored Dec 11, 2023
  
  78b59d73
- [`core`] Make AutoAWQ fused modules compatible with HF transformers (#244) · ae24f424
  Younes Belkada authored Dec 11, 2023
  
  ae24f424
08 Dec, 2023 2 commits
- Load on CPU to avoid OOM (#236) · 6b5dc29f
  Casper authored Dec 08, 2023
  
  6b5dc29f
- Fixed an issue where the Qwen model had too much error after quantization (#243) · 5eb1d2f0
  jundolc authored Dec 08, 2023
  
  5eb1d2f0
03 Dec, 2023 2 commits
- Pass `model_init_kwargs` to `check_and_get_model_type` function (#232) · 5a673bf8
  정한 Rycont authored Dec 03, 2023
  
  5a673bf8
- Fix position ids (#215) · 4d49ae1f
  Casper authored Dec 03, 2023
  
  4d49ae1f
28 Nov, 2023 1 commit
- New scaling to improve perplexity (#216) · d1112e1c
  Casper authored Nov 28, 2023
  
  d1112e1c
20 Nov, 2023 2 commits
- Robust quantization for Catcher (#209) · 63d2aaec
  Casper authored Nov 20, 2023
  
  63d2aaec
- Add Qwen model (#182) · e440c7ac
  Qing authored Nov 20, 2023
```
Co-authored-by: Casper Hansen <casperbh.96@gmail.com>
```
  e440c7ac
18 Nov, 2023 2 commits
- Add config to Base model (#207) · 84c87877
  Casper authored Nov 18, 2023
  
  84c87877
- Fix MPT (#206) · 0e77dbc1
  Casper authored Nov 18, 2023
  
  0e77dbc1
16 Nov, 2023 2 commits
- Bump to 0.1.7 · 87350fef
  Casper authored Nov 16, 2023
  
  87350fef
- Fixed multi-GPU quantization (#196) · 6f516b8d
  Casper authored Nov 16, 2023
  
  6f516b8d
15 Nov, 2023 2 commits
- [`core`] Add `is_hf_transformers` flag (#195) · 74d0fe44
  Younes Belkada authored Nov 15, 2023
  
  74d0fe44
- [`core`] Replace `QuantLlamaMLP` with `QuantFusedMLP` (#188) · 3b362c0d
  Younes Belkada authored Nov 15, 2023
  
  3b362c0d
14 Nov, 2023 1 commit
- Fix multi-GPU loading and inference (#190) · 09c73fb2
  Casper authored Nov 14, 2023
  
  09c73fb2
11 Nov, 2023 4 commits
- Fix cache util logic (#186) · 299c460b
  Casper authored Nov 11, 2023
  
  299c460b
- Pass arguments to AutoConfig (#97) · 7c976752
  Andrey Glushenkov authored Nov 11, 2023
```
Co-authored-by: Casper <casperbh.96@gmail.com>
```
  7c976752
- Adaptive batch sizing (#181) · c5581b27
  Casper authored Nov 11, 2023
  
  c5581b27
- Reset cache on new generation (#178) · df909e83
  Casper authored Nov 11, 2023
  
  df909e83
10 Nov, 2023 2 commits
- New logic for passing past_key_value (#177) · 5db86ec5
  Younes Belkada authored Nov 10, 2023
  
  5db86ec5
- Make sure to delete dummy model (#180) · 1f6f5155
  Casper authored Nov 10, 2023
  
  1f6f5155