GitLab

* Do not shift context for sliding window models

* truncate prompt > 2/3 tokens

* only target gemma2

* fix embedding by adding fixes from llama.cpp upstream

* remove assert

---------
Co-authored-by: Jesper Ek <deadbeef84@gmail.com>

* Revert "use `int32_t` for call to tokenize (#4738)"

This reverts commit 763bb65d.

* Revert "vocab only"

This reverts commit bf54c845.

* Revert "use ffi for tokenizing/detokenizing"

This reverts commit 26a00a04.

This doesn't expose a UX yet, but wires the initial server portion
of progress reporting during load

* feat: enable flash attention if supported

* feat: enable flash attention if supported

* feat: enable flash attention if supported

* feat: add flash_attn support

* Bump llama.cpp to b2761

* Adjust types for bump

* parse wide argv characters on windows

* cleanup

* move cleanup to end of `main`

This should resolve a number of memory leak and stability defects by allowing
us to isolate llama.cpp in a separate process and shutdown when idle, and
gracefully restart if it has problems.  This also serves as a first step to be
able to run multiple copies to support multiple models concurrently.

The release just before ggml-cuda.cu refactoring