GitLab

Go version 1.22.6 is triggering AV false positives, so go back to 1.22.5

* load on empty input

* no load on invalid input

add conversion for microsoft phi 3 mini/medium 4k, 128k

* simplify input output

* direct comp

* in line image

* rm error pointer type

* update response testing

* lint

This reverts commit 8aac2243.

For simplicity, perform parallelization of embedding requests in the API handler instead of offloading this to the subprocess runner. This keeps the scheduling story simpler as it builds on existing parallel requests, similar to existing text completion functionality.

Don't allow loading models that would lead to memory exhaustion (across vram, system memory and disk paging). This check was already applied on Linux but should also be applied on Windows as well.

server/download.go: Fix a typo in log

Set *.png and *.ico to be treated as binary files.

removeall to remove non-empty temp dirs

This partially reverts b732beba. It
seems like explicitly setting all files to use line feeds was done due
to issues with the go linter, hence it can be restricted to those files
(https://github.com/ollama/ollama/pull/6235#issuecomment-2278745953).

Don't hard fail on sparse setup error

It seems this can fail in some casees, but proceed
with the download anyway.

Harden intel boostrap for nil pointers

Adjust arm cuda repo paths

Catch one more error log

Store layers inside manifests consistently as values.

Signed-off-by: Jitang Lei <leijitang@outlook.com>

Commit 1829fb61 ("manifest: Fix crash on startup when trying to clean up
unused files (#5840)") changed the config layer stored in manifests
from a pointer to a value. This was done in order to avoid potential
nil pointer dereferences after it is deserialized from JSON in the
event that the field is missing.

This changes the Layers slice to also be stored by value. This enables
consistency in handling across the two objects.

When creating a model the config layer is appended to the list of
layers and then the last layer is used as the config when writing the
manifest. This change directly uses the config layer to write the
manifest. There is no behavior change but it is less error prone.

* Use llama3.1 in tools example

* Update api.md

Fix crash on startup when trying to clean up unused files (#5840)

Currently if the config field is missing in the manifest file (or
corrupted), Ollama will crash when it tries to read it. This can
happen at startup or when pulling new models.

This data is mostly just used for showing model information so we
can be tolerant of it not being present - it is not required to
run the models. Besides avoiding crashing, this also gives us the
ability to restructure the config in the future by pulling it
into the main manifest file.

The change b732beba makes all files text files and sets lf as eol. This
will automatically change all files to have lf if they are touched by
git (e.g. via git status). This change cannot be stashed and makes it
hard to work with the repo (rebase and checkout don't really work). See
also #6183.

Here, we set the offending files (*.png and *.ico, but that might be
more in the future) to be treated as binary files and not be changed by
git.

If there is an error when opening a manifest file (corrupted, permission denied, etc.)
then the referenced layers will not be included in the list of active
layers. This causes them to be deleted when pruning happens at startup
or a model is pulled.

In such a situation, we should prefer to preserve data in the hopes that
it can be recovered rather than being agressive about deletion.

Ensure sparse files on windows during download

The file.Truncate call on windows will write the whole file
unless you set the sparse flag, leading to heavy I/O at the
beginning of download.  This should improve our
I/O behavior on windows and put less stress on the users disk.