"examples/vscode:/vscode.git/clone" did not exist on "a71e3a401e4a68d23900ff7d3d0437676392fa19"
convert: quantize from safetensors needs kv (#10675)
When creating a quantized model from safetensors we need the array KV values to be loaded.Changing this value to -1 loads the KV values on the returned layer to be used and saved during quantization.
Showing
Please register or sign in to comment