Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "73f73120fed1bc806d7487dcc4e999ad63e12947"
Unverified Commit 3b5a4a77 authored by royjhan's avatar royjhan Committed by GitHub
Browse files

Return Correct Prompt Eval Count Regardless of Cache Prompt (#5371) 

* openai compatibility

* Revert "openai compatibility"

This reverts commit d3f98a811e00fc497d889c8c45b0cfec5b64690c.

* remove erroneous subtraction of prompt cache
parent daed0634
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
llm/ext_server/server.cpp
View file @ 3b5a4a77
...@@ -1732,7 +1732,7 @@ struct llama_server_context ...@@ -1732,7 +1732,7 @@ struct llama_server_context
slot.n_past -= 1; slot.n_past -= 1;
} }
slot.n_prompt_tokens_processed = slot.n_prompt_tokens - slot.n_past; slot.n_prompt_tokens_processed = slot.n_prompt_tokens;
if (slot.ga_n != 1) if (slot.ga_n != 1)
{ {
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment