GitLab

Store layers inside manifests consistently as values.

Commit 1829fb61 ("manifest: Fix crash on startup when trying to clean up
unused files (#5840)") changed the config layer stored in manifests
from a pointer to a value. This was done in order to avoid potential
nil pointer dereferences after it is deserialized from JSON in the
event that the field is missing.

This changes the Layers slice to also be stored by value. This enables
consistency in handling across the two objects.

When creating a model the config layer is appended to the list of
layers and then the last layer is used as the config when writing the
manifest. This change directly uses the config layer to write the
manifest. There is no behavior change but it is less error prone.

* Use llama3.1 in tools example

* Update api.md

Fix crash on startup when trying to clean up unused files (#5840)

Currently if the config field is missing in the manifest file (or
corrupted), Ollama will crash when it tries to read it. This can
happen at startup or when pulling new models.

This data is mostly just used for showing model information so we
can be tolerant of it not being present - it is not required to
run the models. Besides avoiding crashing, this also gives us the
ability to restructure the config in the future by pulling it
into the main manifest file.

If there is an error when opening a manifest file (corrupted, permission denied, etc.)
then the referenced layers will not be included in the list of active
layers. This causes them to be deleted when pruning happens at startup
or a model is pulled.

In such a situation, we should prefer to preserve data in the hopes that
it can be recovered rather than being agressive about deletion.

Ensure sparse files on windows during download

The file.Truncate call on windows will write the whole file
unless you set the sparse flag, leading to heavy I/O at the
beginning of download.  This should improve our
I/O behavior on windows and put less stress on the users disk.

fix concurrency test

Implement linux NUMA detection

use testing tempdirs

If the system has multiple numa nodes, enable numa support in llama.cpp
If we detect numactl in the path, use that, else use the basic "distribute" mode.

line feed

Users using `journalctl` to get logs for issue logging sometimes don't realize that paging is causing information to be missed.

enable gofmt/gofumpt/goimports/tenv

* Update docs

* token bug corrected

* Update docs/openai.md

* Update docs/openai.md

* add suffix

* merge conflicts

* merge conflicts

* docs without usage

* no usage

* rm metric note

* add prompt tokens to embed response

* rm slog

* metrics

* types

* prompt n

* clean up

* reset submodule

* add tokens to v1/embeddings

* separate usage

* OpenAI Docs

* Update docs/openai.md
Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>

* Remove newline

---------
Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>

Fix context in /api/generate grows too much (#5980).

refactor convert