GitLab

This records more GPU usage information for eventual UX inclusion.

- FROM /path/to/{safetensors,pytorch}
- FROM /path/to/fp{16,32}.bin
- FROM model:fp{16,32}

Trying to live off the land for cuda libraries was not the right strategy.  We need to use the version we compiled against to ensure things work properly

* fix llava models not working after first request

* individual requests only for llava models

This moves all the env var reading into one central module
and logs the loaded config once at startup which should
help in troubleshooting user server logs

log when the waiting for the process to stop to help debug when other tasks execute during this wait.
expire timer clear the timer reference because it will not be reused.
close will clean up expireTimer if calling code has not already done this.

* Bump llama.cpp to b2761

* Adjust types for bump

This will speed up CI which already tries to only build static for unit tests

This will make it simpler for CI to accumulate artifacts from prior steps

* llm: limit generation to 10x context size to avoid run on generations

* add comment

* simplify condition statement

If we get our predictions wrong, this can be used to
set a lower memory limit as a workaround.  Recent multi-gpu
refactoring accidentally removed it, so this adds it back.

Now that the llm runner is an executable and not just a dll, more users are facing
problems with security policy configurations on windows that prevent users
writing to directories and then executing binaries from the same location.
This change removes payloads from the main executable on windows and shifts them
over to be packaged in the installer and discovered based on the executables location.
This also adds a new zip file for people who want to "roll their own" installation model.

Tmp cleaners can nuke the file out from underneath us.  This detects the missing
runner, and re-initializes the payloads.