GitLab

This should resolve the problem where we don't fully unload from the GPU
when we go idle.

This focuses on Windows first, but coudl be used for Mac
and possibly linux in the future.

This reverts commit 6920964b.

This reverts commit 2f9ed52b.

This wires up some new logic to start using sysfs to discover AMD GPU
information and detects old cards we can't yet support so we can fallback to CPU mode.

Make sure that when a shutdown signal comes, we shutdown quickly instead
of waiting for a potentially long exchange to wrap up.

When we store our libraries in a temp dir, a reaper might clean
them when we are idle, so make sure to check for them before
we reload.

Only apply patches if we have any, and make sure to cleanup
every file we patched at the end to leave the tree clean

This requires an upstream change to support graceful termination,
carried as a patch.

* Fix clearing kv cache between requests with the same prompt

* fix powershell script

For old macs, accelerate seems to cause crashes, but for
AVX2 capable macs, it does not.

This wires up logging in llama.cpp to always go to stderr, and also
turns up logging if OLLAMA_DEBUG is set.

Detect potential error scenarios so we can fallback to CPU mode without
hitting asserts.

The linux build now support parallel CPU builds to speed things up.
This also exposes AMD GPU targets as an optional setting for advaced
users who want to alter our default set.

Update gpu.go initGPUHandles() to declare gpuHandles variable before
reading it. This resolves an "invalid memory address or nil pointer
dereference" error.

Update dyn_ext_server.c to avoid setting the RTLD_DEEPBIND flag under
__TERMUX__ (Android).

A few obvious levels were adjusted, but generally everything mapped to "info" level.

This also refines the build process for the ext_server build.

Upstream llama.cpp has added a new dependency with the
NVIDIA CUDA Driver Libraries (libcuda.so) which is part of the
driver distribution, not the general cuda libraries, and is not
available as an archive, so we can not statically link it.  This may
introduce some additional compatibility challenges which we'll
need to keep an eye on.