GitLab

Make the build faster

Enhanced GPU discovery and multi-gpu support with concurrency

While models are loading, the VRAM metrics are dynamic, so try
to load on a GPU that doesn't have a model actively loading, or wait
to avoid races that lead to OOMs

This library will give us the most reliable free VRAM reporting on windows
to enable concurrent model scheduling.

adjust timing on some tests so they don't timeout on small/slow GPUs

Our default behavior today is to try to fit into a single GPU if possible.
Some users would prefer the old behavior of always spreading across
multiple GPUs even if the model can fit into one.  This exposes that
tunable behavior.

Still not complete, needs some refinement to our prediction to understand the
discrete GPUs available space so we can see how many layers fit in each one
since we can't split one layer across multiple GPUs we can't treat free space
as one logical block

This worked remotely but wound up trying to spawn multiple servers
locally which doesn't work

Now that we call the GPU discovery routines many times to
update memory, this splits initial discovery from free memory
updating.

The amdgpu drivers free VRAM reporting omits some other apps, so leverage the
upstream DRM driver which keeps better tabs on things

This reverts commit 476fb8e8.

Actually skip PhysX on windows

fix: multibyte utf16

Revert "proper utf16 support"

This reverts commit 66ab4877.

this change broke utf-8 scanning of multi-byte runes

fix: multiple templates when creating from model

multiple templates may appear in a model if a model is created from
another model that 1) has an autodetected template and 2) defines a
custom template

Revert "Merge pull request #4938 from ollama/mxyng/fix-byte-order"

This reverts commit f5f245cc, reversing
changes made to 94d37fdc.

this change broke gguf v2 which is incorrectly detected as big endian

proper utf16 support

update import.md

fix: skip removing layers that no longer exist