GitLab

* Revert "fix cmake build (#5505)"

This reverts commit 4fd5f352.

* llm: fix missing dylibs by restoring old build behavior

* crlf -> lf

* Use common prefix to select slot

* actually report `longest`

* openai compatibility

* Revert "openai compatibility"

This reverts commit d3f98a811e00fc497d889c8c45b0cfec5b64690c.

* remove erroneous subtraction of prompt cache

* Do not shift context for sliding window models

* truncate prompt > 2/3 tokens

* only target gemma2

* fix embedding by adding fixes from llama.cpp upstream

* remove assert

---------
Co-authored-by: Jesper Ek <deadbeef84@gmail.com>

* Revert "use `int32_t` for call to tokenize (#4738)"

This reverts commit 763bb65d.

* Revert "vocab only"

This reverts commit bf54c845.

* Revert "use ffi for tokenizing/detokenizing"

This reverts commit 26a00a04.

This doesn't expose a UX yet, but wires the initial server portion
of progress reporting during load

* feat: enable flash attention if supported

* feat: enable flash attention if supported

* feat: enable flash attention if supported

* feat: add flash_attn support

* Bump llama.cpp to b2761

* Adjust types for bump

* parse wide argv characters on windows

* cleanup

* move cleanup to end of `main`

This should resolve a number of memory leak and stability defects by allowing
us to isolate llama.cpp in a separate process and shutdown when idle, and
gracefully restart if it has problems.  This also serves as a first step to be
able to run multiple copies to support multiple models concurrently.

The release just before ggml-cuda.cu refactoring

Signed-off-by: racerole <jiangyifeng@outlook.com>