GitLab

Bump llama.cpp to b2737

Fix exe name for zip packaging on windows

The zip file encodes the OS and architecture, so keep the short exe name

Refactor windows generate for more modular usage

Move cuda/rocm dependency gathering into generate script

This will make it simpler for CI to accumulate artifacts from prior steps

Fix release CI

download-artifact path was being used incorrectly.  It is where to
extract the zip not the files in the zip to extract.  Default is
workspace dir which is what we want, so omit it

only count output tensors

Improve mac parallel performance

* reload model if `num_gpu` changes

* dont reload on -1

* fix tests

* llm: limit generation to 10x context size to avoid run on generations

* add comment

* simplify condition statement

Move ggml loading to when attempting to fit

update copy handler to use model.Name

fix: from blob

AMD gfx patch rev is hex

Report errors on server lookup instead of path lookup failure

Correctly handle gfx90a discovery

This allows users of a valid Digest to know it has a minimum of 2
characters in the hash part for use when sharding.

This is a reasonable restriction as the hash part is a SHA256 hash which
is 64 characters long, which is the common hash used. There is no
anticipation of using a hash with less than 2 characters.

Also, add MustParseDigest.

Also, replace Digest.Type with Digest.Split for getting both the type
and hash parts together, which is most the common case when asking for
either.

Add back memory escape valve

If we get our predictions wrong, this can be used to
set a lower memory limit as a workaround.  Recent multi-gpu
refactoring accidentally removed it, so this adds it back.

Move nested payloads to installer and zip file on windows

Give the go routine a moment to deliver the expired event