• pppppM's avatar
    [Feature] Support AWQ (#108) · d3dbe179
    pppppM authored
    * support kv cache offload
    
    * add dataloader docstring
    
    * complete gitignore
    
    * refactor collect mod fn
    
    * add calibration
    
    * fix lint
    
    * add observers and quantizers
    
    * fix lints
    
    * add global available mixin
    
    * fix lints
    
    * split batch inference
    
    * support smoothquant and awq
    
    * update export kv scales
    
    * fix lints
    
    * fix some bugs
    
    * update weight only usage
    
    * update usage
    
    * auto mapping and support smooth internlm
    
    * trust remote code
    
    * fix num head key error
    
    * fix bias error
    
    * align shape and pack order with llm-awq
    
    * modified according to LZHgrla's comments.
    
    * update gitignore
    
    * fix kv qparams export error
    
    * update usage
    
    * decouple calibrate and awq
    
    * update docstrings
    
    * update api name
    
    * update readme
    
    * update readme
    
    * update readme
    
    * update readme
    
    * update kv_qparams and readme
    
    * fix typos
    d3dbe179
.gitignore 781 Bytes