Return Correct Prompt Eval Count Regardless of Cache Prompt (#5371)

* openai compatibility * Revert "openai compatibility" This reverts commit d3f98a811e00fc497d889c8c45b0cfec5b64690c. * remove erroneous subtraction of prompt cache

Return Correct Prompt Eval Count Regardless of Cache Prompt (#5371)
* openai compatibility * Revert "openai compatibility" This reverts commit d3f98a811e00fc497d889c8c45b0cfec5b64690c. * remove erroneous subtraction of prompt cache
3b5a4a77 · royjhan · GitHub · daed0634 · 3b5a4a77
Unverified Commit 3b5a4a77 authored Jul 03, 2024 by royjhan Committed by GitHub Jul 03, 2024
Show whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

llm/ext_server/server.cpp llm/ext_server/server.cpp +1 -1

No files found.
--- a/llm/ext_server/server.cpp
+++ b/llm/ext_server/server.cpp
@@ -1732,7 +1732,7 @@ struct llama_server_context
                            slot.n_past -= 1;
                        }

-                        slot.n_prompt_tokens_processed = slot.n_prompt_tokens - slot.n_past;
+                        slot.n_prompt_tokens_processed = slot.n_prompt_tokens;

                        if (slot.ga_n != 1)
                        {