GitLab

Move cuda/rocm dependency gathering into generate script

This will make it simpler for CI to accumulate artifacts from prior steps

Fix release CI

download-artifact path was being used incorrectly.  It is where to
extract the zip not the files in the zip to extract.  Default is
workspace dir which is what we want, so omit it

only count output tensors

Improve mac parallel performance

* reload model if `num_gpu` changes

* dont reload on -1

* fix tests

* llm: limit generation to 10x context size to avoid run on generations

* add comment

* simplify condition statement

Move ggml loading to when attempting to fit

update copy handler to use model.Name

fix: from blob

AMD gfx patch rev is hex

Report errors on server lookup instead of path lookup failure

Correctly handle gfx90a discovery

This allows users of a valid Digest to know it has a minimum of 2
characters in the hash part for use when sharding.

This is a reasonable restriction as the hash part is a SHA256 hash which
is 64 characters long, which is the common hash used. There is no
anticipation of using a hash with less than 2 characters.

Also, add MustParseDigest.

Also, replace Digest.Type with Digest.Split for getting both the type
and hash parts together, which is most the common case when asking for
either.

Add back memory escape valve

If we get our predictions wrong, this can be used to
set a lower memory limit as a workaround.  Recent multi-gpu
refactoring accidentally removed it, so this adds it back.

Move nested payloads to installer and zip file on windows

Give the go routine a moment to deliver the expired event

Now that the llm runner is an executable and not just a dll, more users are facing
problems with security policy configurations on windows that prevent users
writing to directories and then executing binaries from the same location.
This change removes payloads from the main executable on windows and shifts them
over to be packaged in the installer and discovered based on the executables location.
This also adds a new zip file for people who want to "roll their own" installation model.

Detect and recover if runner removed

adding phi-3 mini to readme

Tmp cleaners can nuke the file out from underneath us.  This detects the missing
runner, and re-initializes the payloads.

Adds support for customizing GPU build flags in llama.cpp