GitLab

On some systems, 1 minute isn't sufficient to finish the load after it
hits 100% This creates 2 distinct timers, although they're both set to
the same value for now so we can refine the timeouts further.

Signed-off-by: Lei Jitang <leijitang@outlook.com>

If the client closes the connection before we finish loading the model
we abort, so lets make the log message clearer why to help users
understand this failure mode

Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

This doesn't expose a UX yet, but wires the initial server portion
of progress reporting during load

Co-authored-by: ManniX-ITA <20623405+mann1x@users.noreply.github.com>

* put flash attention behind flag for now

* add test

* remove print

* up timeout for sheduler tests

* feat: enable flash attention if supported

* feat: enable flash attention if supported

* feat: enable flash attention if supported

* feat: add flash_attn support

Windows already implements these, carry over to linux.

Only dump env vars we care about in the logs

This reverts commit ce3b212d.

* dont clamp ctx size in `PredictServerFit`

* minimum 4 context

* remove context warning

This records more GPU usage information for eventual UX inclusion.

This will bubble up a much more informative error message if noexec
is preventing us from running the subprocess