GitLab

- only reload the running llm if the model has changed, or the options for loading the running model have changed
- rename loaded llm to runner to differentiate from loaded model image
- remove logic which keeps the first system prompt in the generation context

- remove new lines from llama.cpp error messages relayed to client
- check api option types and return error on wrong type
- change num layers from 95% VRAM to 92% VRAM

* update streaming request accept header
* add optional stream param to request bodies

* include seed in params for llama.cpp server and remove empty filter for temp

* relay default predict options to llama.cpp

- reorganize options to match predict request for readability

* omit empty stop

---------
Co-authored-by: hallh <hallh@users.noreply.github.com>

* linux gpu support
* handle multiple gpus
* add cuda docker image (#488)
---------
Co-authored-by: Michael Yang <mxyng@pm.me>

* remove c code
* pack llama.cpp
* use request context for llama_cpp
* let llama_cpp decide the number of threads to use
* stop llama runner when app stops
* remove sample count and duration metrics
* use go generate to get libraries
* tmp dir for running llm

* cmd: support OLLAMA_HOST environment variable

This commit adds support for the OLLAMA_HOST environment
variable. This variable can be used to specify the host to which
the client should connect. This is useful when the client is
running somewhere other than the host where the server is running.

The new api.FromEnv function is used to read configure clients from the
environment. Clients wishing to use the environment variable being
consistent with the Ollama CLI can use this new function.

* Update api/client.go
Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>

* Update api/client.go
Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>

---------
Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>

Co-Authored-By: Jeffrey Morgan <jmorganca@gmail.com>

- default to embeddings enabled
- move embedding logic for loaded model to request
- allow embedding full directory
- close llm on reload

Fixes #297
Fixes #296

num_keep defines how many tokens to keep in the context when truncating
inputs. if left to its default value of -1, the server will calculate
num_keep to be the left of the system instructions

- read runner options from map to see what was specified explicitly and overwrite zero values