1. 11 Aug, 2023 1 commit
    • pppppM's avatar
      [Feature] Support AWQ (#108) · d3dbe179
      pppppM authored
      * support kv cache offload
      
      * add dataloader docstring
      
      * complete gitignore
      
      * refactor collect mod fn
      
      * add calibration
      
      * fix lint
      
      * add observers and quantizers
      
      * fix lints
      
      * add global available mixin
      
      * fix lints
      
      * split batch inference
      
      * support smoothquant and awq
      
      * update export kv scales
      
      * fix lints
      
      * fix some bugs
      
      * update weight only usage
      
      * update usage
      
      * auto mapping and support smooth internlm
      
      * trust remote code
      
      * fix num head key error
      
      * fix bias error
      
      * align shape and pack order with llm-awq
      
      * modified according to LZHgrla's comments.
      
      * update gitignore
      
      * fix kv qparams export error
      
      * update usage
      
      * decouple calibrate and awq
      
      * update docstrings
      
      * update api name
      
      * update readme
      
      * update readme
      
      * update readme
      
      * update readme
      
      * update kv_qparams and readme
      
      * fix typos
      d3dbe179