GitLab

also removes stale model dir instructions for windows

The current scheduler algorithm of picking the paralellism based on available
VRAM complicates the upcoming dynamic layer memory allocation algorithm.  This
changes the default to 1, with the intent going forward that parallelism is
explicit and will no longer be dynamically determined.  Removal of the dynamic
logic will come in a follow up.

* API: expose context size of loaded models

* CLI: add context UX

This adds a column in the ps output to show the models context size.

switch a few constants to variables

The root cause was an unclean upgrade - this code is fine.

This reverts commit 45f216a9.

usage example:
  go test --tags=integration,perf -count 1 ./integration -v -timeout 1h -run TestModelsPerf 2>&1 | tee int.log
  cat int.log | grep MODEL_PERF_HEADER | cut -f2- -d: > perf.csv
  cat int.log | grep MODEL_PERF_DATA | cut -f2- -d: >> perf.csv

Favor the dmg now.

This adds some extra logs to make the new engine a bit more consistent
with the llama engine.

This is causing segfaults, so disable it. Currently UUIDs are only
used for debugging purposes, although they planned to be used in
additional ways in the future.

Bug #11211

this tensor isn't compatible with cuda when quantized to q4_K so skip it

* update patches

* cherry pick metal mean kernel

* cherry pick cuda mean kernel

* gemma3n

Glue the rocm and archive builds back together.

ggml: fix crash for array head counts

The preset CMAKE_HIP_FLAGS isn't getting used on Windows.
This passes the parallel flag in through the C/CXX flags, along
with suppression for some log spew warnings to quiet down the build.

* CI: switch windows to vs 2022

* ci: fix regex match

For smaller context models, make sure we do not exceed the training size.

* Re-remove cuda v11

Revert the revert - drop v11 support requiring drivers newer than Feb 23

This reverts commit c6bcdc42.

* Simplify layout

With only one version of the GPU libraries, we can simplify things down somewhat.  (Jetsons still require special handling)

* distinct sbsa variant for linux arm64

This avoids accidentally trying to load the sbsa cuda libraries on
a jetson system which results in crashes.

* temporary prevent rocm+cuda mixed loading

Enable parallel building of the GPU architectures.

* Reapply "feat: incremental gguf parser (#10822)" (#11114)

This reverts commit a6e64fbd.

* fix older ggufs

We don't check the return status after computing the graph, which
can silently lead to bad outputs if we try to keep going and future
computation succeeds. This appears to happens in certain cases on
Apple M2 devices.

Fixes #11070

Verified these fail on 0.9.1 and pass on HEAD.