GitLab

* Update api.md

Changed the calculation of tps (token/s) in the documentation

* Update docs/api.md

---------
Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>

This also bumps up the default to be 50 queued requests
instead of 10.

* Update 'llama2' -> 'llama3' in most places

---------
Co-authored-by: Patrick Devine <patrick@infrahq.com>

replace CRLF with LF

Updated the code a bit

Fixed a typo

Changed ollama.call() for ollama.invoke() as per deprecated documentation from langchain

Remove Modelfile parameters that are decided at runtime

This commit introduces a more friendly way to build Ollama dependencies
and the binary without abusing `go generate` and removing the
unnecessary extra steps it brings with it.

This script also provides nicer feedback to the user about what is
happening during the build process.

At the end, it prints a helpful message to the user about what to do
next (e.g. run the new local Ollama).

Fixes gh-3514
Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>

We may have users that run into problems with our current
payload model, so this gives us an escape valve.

The recent ROCm change partially removed idempotent
payloads, but the ggml-metal.metal file for mac was still
idempotent.  This finishes switching to always extract
the payloads, and now that idempotentcy is gone, the
version directory is no longer useful.

This refines where we extract the LLM libraries to by adding a new
OLLAMA_HOME env var, that defaults to `~/.ollama` The logic was already
idempotenent, so this should speed up startups after the first time a
new release is deployed.  It also cleans up after itself.

We now build only a single ROCm version (latest major) on both windows
and linux.  Given the large size of ROCms tensor files, we split the
dependency out.  It's bundled into the installer on windows, and a
separate download on windows.  The linux install script is now smart and
detects the presence of AMD GPUs and looks to see if rocm v6 is already
present, and if not, then downloads our dependency tar file.

For Linux discovery, we now use sysfs and check each GPU against what
ROCm supports so we can degrade to CPU gracefully instead of having
llama.cpp+rocm assert/crash on us.  For Windows, we now use go's windows
dynamic library loading logic to access the amdhip64.dll APIs to query
the GPU information.