llm/ext_server/server.cpp · 3b5a4a77f3a191e368af3412e5de9b38b4f80771 · OpenDAS / ollama

Return Correct Prompt Eval Count Regardless of Cache Prompt (#5371) · 3b5a4a77

royjhan authored Jul 03, 2024

* openai compatibility

* Revert "openai compatibility"

This reverts commit d3f98a811e00fc497d889c8c45b0cfec5b64690c.

* remove erroneous subtraction of prompt cache

3b5a4a77

server.cpp 123 KB

Replace server.cpp