"vscode:/vscode.git/clone" did not exist on "132649c4e577e4c133d41c910549ad4480e44822"
convert: quantize from safetensors needs kv (#10675)
When creating a quantized model from safetensors we need the array KV values to be loaded.Changing this value to -1 loads the KV values on the returned layer to be used and saved during quantization.
Showing
Please register or sign in to comment