• Jesse Gross's avatar
    ggml: Use assigned layers when reporting loading stats · acef9b4c
    Jesse Gross authored
    Reporting params.NumGPULayers can be misleading because it is the
    requested number of layers, not the actual number that is loaded.
    While they are often the same, there are cases where they might mismatch,
    such as if the GPU backend is missing.
    acef9b4c
ggml.go 34.5 KB