index.md

(quantization-index)=

# Quantization

Quantization trades off model precision for smaller memory footprint, allowing large models to be run on a wider range of devices.

:::{toctree}
:caption: Contents
:maxdepth: 1

supported_hardware
auto_awq
bnb
gguf
int4
int8
fp8
quantized_kvcache
:::