server/text_generation_server/layers/gptq/quantize.py · 8ec57558cd5b7b2ad3eaacdd6295a3db0c9092d0 · OpenDAS / text-generation-inference · GitLab

Find file Blame History Permalink

server quantize: store quantizer config in standard format (#2299) · 53aec273
Daniël de Kok authored Jul 30, 2024
```
- Create `quantization_config` option in the model config.
- Don't store the quantizer config in tensors anymore.
```
53aec273

quantize.py 31.3 KB