GitLab

This doesn't expose a UX yet, but wires the initial server portion
of progress reporting during load

Co-authored-by: ManniX-ITA <20623405+mann1x@users.noreply.github.com>

* put flash attention behind flag for now

* add test

* remove print

* up timeout for sheduler tests

* feat: enable flash attention if supported

* feat: enable flash attention if supported

* feat: enable flash attention if supported

* feat: add flash_attn support

Windows already implements these, carry over to linux.

Only dump env vars we care about in the logs

This reverts commit ce3b212d.

* dont clamp ctx size in `PredictServerFit`

* minimum 4 context

* remove context warning

This records more GPU usage information for eventual UX inclusion.

This will bubble up a much more informative error message if noexec
is preventing us from running the subprocess

- FROM /path/to/{safetensors,pytorch}
- FROM /path/to/fp{16,32}.bin
- FROM model:fp{16,32}

Trying to live off the land for cuda libraries was not the right strategy.  We need to use the version we compiled against to ensure things work properly

* fix llava models not working after first request

* individual requests only for llava models

This moves all the env var reading into one central module
and logs the loaded config once at startup which should
help in troubleshooting user server logs