"git@developer.sourcefind.cn:OpenDAS/torchaudio.git" did not exist on "2e58f18acf0d672ef1214d7388e6dabed3b5723f"
Commit ef549d51 authored by Jesse Gross's avatar Jesse Gross Committed by Jesse Gross
Browse files

ggml: Increase maximum graph size

The initial implementation of qwen3-vl:235b exceeded the maximum graph
size based on the number of tensors. Although this was later fixed
through the use of the mrope operation, we are close to the limit in
some cases. This updates to track the current llama.cpp usage of GGML.
parent d2158ca6
...@@ -378,7 +378,7 @@ func New(modelPath string, params ml.BackendParams) (ml.Backend, error) { ...@@ -378,7 +378,7 @@ func New(modelPath string, params ml.BackendParams) (ml.Backend, error) {
} }
} }
maxGraphNodes := max(8192, len(meta.Tensors().Items())*5) maxGraphNodes := max(1024, len(meta.Tensors().Items())*8)
sched := C.ggml_backend_sched_new_ext( sched := C.ggml_backend_sched_new_ext(
(*C.ggml_backend_t)(unsafe.Pointer(&schedBackends[0])), (*C.ggml_backend_t)(unsafe.Pointer(&schedBackends[0])),
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment