GitLab

* put flash attention behind flag for now

* add test

* remove print

* up timeout for sheduler tests

The spelling of the term "request" has been corrected, which was previously mistakenly written as "requeset" in the error log message.

particularly useful for zipfiles and f16s

* Fixed the API endpoint /api/tags to return {models: []} instead of {models: null} when the model list is empty.

* Update server/routes.go

---------
Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>

The APIs we query are optimistic on free space, and windows pages
VRAM, so we don't have to wait to see reported usage recover on unload

This reverts commit 0fec3525.

Make sure the first GPU has the most free space

* rename `--quantization` to `--quantize`

* backwards

* Update api/types.go
Co-authored-by: Michael Yang <mxyng@pm.me>

---------
Co-authored-by: Michael Yang <mxyng@pm.me>

* dont clamp ctx size in `PredictServerFit`

* minimum 4 context

* remove context warning

Ensure the runners are terminated

The GPU drivers take a while to update their free memory reporting, so we need
to wait until the values converge with what we're expecting before proceeding
to start another runner in order to get an accurate picture.

This cleans up the logging for GPU discovery a bit, and can
serve as a foundation to report GPU information in a future UX.