• Jesse Gross's avatar
    llamarunner: Init GGML before printing system info · 010313bb
    Jesse Gross authored
    We currently print system info before the GGML backends are loaded.
    This results in only getting information about the default lowest
    common denominator runner. If we move up the GGML init then we can
    see what we are actually running.
    
    Before:
    time=2025-02-14T11:15:07.606-08:00 level=INFO source=runner.go:935 msg=system info="CPU : LLAMAFILE = 1 | CPU : LLAMAFILE = 1 | cgo(gcc)" threads=24
    
    After:
    time=2025-02-14T11:16:02.936-08:00 level=INFO source=runner.go:935 msg=system info="CPU : LLAMAFILE = 1 | CPU : LLAMAFILE = 1 | CUDA : ARCHS = 890 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | LLAMAFILE = 1 | cgo(gcc)" threads=24
    010313bb
runner.go 25.5 KB