Skip to content

GitLab

Unverified Commit 4458efb7 authored by Jeffrey Morgan's avatar Jeffrey Morgan Committed by GitHub
Browse files

Load all layers on `arm64` macOS if model is small enough (#2149)

parent ceea5994
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 1 deletion
llm/llm.go
View file @ 4458efb7
...@@ -70,7 +70,8 @@ func New(workDir, model string, adapters, projectors []string, opts api.Options) ...@@ -70,7 +70,8 @@ func New(workDir, model string, adapters, projectors []string, opts api.Options)
break break
} }
opts.NumGPU = 1 // TODO: implement layer splitting on macOS
opts.NumGPU = 999
default: default:
if info.Library == "cpu" { if info.Library == "cpu" {
slog.Info("GPU not available, falling back to CPU") slog.Info("GPU not available, falling back to CPU")
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment