GitLab

If the client closes the connection before we finish loading the model
we abort, so lets make the log message clearer why to help users
understand this failure mode

Tidy up developer guide a little

Wire up load progress

This doesn't expose a UX yet, but wires the initial server portion
of progress reporting during load

Co-authored-by: ManniX-ITA <20623405+mann1x@users.noreply.github.com>

* put flash attention behind flag for now

* add test

* remove print

* up timeout for sheduler tests

PreTokenziers -> PreTokenizers

add Ctrl + W shortcut

simplify safetensors reading

Convert directly from llama3

The spelling of the term "request" has been corrected, which was previously mistakenly written as "requeset" in the error log message.

fix quantize file types

cache and reuse intermediate blobs

* feat: enable flash attention if supported

* feat: enable flash attention if supported

* feat: enable flash attention if supported

* feat: add flash_attn support

particularly useful for zipfiles and f16s

Don't return error on signal exit