Skip to content

GitLab

Closed
Created by comey@imwangb

能支持int4量化模型推理么？

lmdeploy/src/turbomind/models/llama/LlamaLinear.h:forwardInt4的接口代码全部被注释了，导致无法运行量化后的模型