GitLab

We build the GPU libraries with AVX enabled to ensure that if not all
layers fit on the GPU we get better performance in a mixed mode.
If the user is using a virtualization/emulation system that lacks AVX
this used to result in an illegal instruction error and crash before this
fix.  Now we will report a warning in the server log, and just use
CPU mode to ensure we don't crash.

fix build

download: add inactivity monitor

Add back ROCm container support

Ignore AMD integrated GPUs

Fix crash on cuda ml init failure

This adds ROCm support back as a discrete image.

Detect and ignore integrated GPUs reported by rocm.

The new driver lookup code was triggering after init failure due to a missing return

Switch back to ubuntu base

The size increase for rocm support in the standard image is problematic
We'll revisit multiple tags for rocm support in a follow up PR.

build cuda and rocm

* Fix clearing kv cache between requests with the same prompt

* fix powershell script

stub generate outputs for lint

refactor tensor read

More logging for gpu management

Fix an ordering glitch of dlerr/dlclose and add more logging to help
root cause some crashes users are hitting. This also refines the
function pointer names to use the underlying function names instead
of simplified names for readability.

Report more information about GPUs in verbose mode

This adds additional calls to both CUDA and ROCm management libraries to
discover additional attributes about the GPU(s) detected in the system, and
wires up runtime verbosity selection.  When users hit problems with GPUs we can
ask them to run with `OLLAMA_DEBUG=1 ollama serve` and share the results.

Set a default version using git describe

If a VERSION is not specified, this will generate a version string that
represents the state of the repo.  For example `0.1.21-12-gffaf52e1-dirty`
representing 12 commits away from 0.1.21 tag, on commit gffaf52e1
and the tree is dirty.

Refine Accelerate usage on mac

For old macs, accelerate seems to cause crashes, but for
AVX2 capable macs, it does not.