• Baber Abbasi's avatar
    [HF] fix quantization config (#3039) · fea4d11d
    Baber Abbasi authored
    * Try fixing issue 3026 which is caused by the quantization_config argument introduced in Commit 758c5ed8
    
    .
    The argument is in Dict type, but for a GPTQ quantized model, it has a conflict with the huggingface interface which expects QuantizationConfigMixin type.
    Current solution is removing quantization_config argument in HFLM._create_model() of lm_eval/models/huggingface.py.
    Require further modification to restore the functionality provided by the previous commit.
    
    * wrap quantization_config in AutoQuantizationConfig
    
    * handle quantization config not dict
    
    * wrap quantization_config in AutoQuantizationConfig if dict
    
    ---------
    Co-authored-by: default avatarshanhx2000 <hs359@duke.edu>
    fea4d11d
huggingface.py 62.2 KB