[Bugfix] Steaming continuous_usage_stats default to False (#9709)

Signed-off-by: Sam Stoelinga <sammiestoel@gmail.com>

[Bugfix] Steaming continuous_usage_stats default to False (#9709)
Signed-off-by: Sam Stoelinga <sammiestoel@gmail.com>
067e77f9 · Sam Stoelinga · GitHub · 6567e137 · 067e77f9
Unverified Commit 067e77f9 authored Oct 25, 2024 by Sam Stoelinga Committed by GitHub Oct 26, 2024
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

vllm/entrypoints/openai/protocol.py vllm/entrypoints/openai/protocol.py +1 -1

No files found.
--- a/vllm/entrypoints/openai/protocol.py
+++ b/vllm/entrypoints/openai/protocol.py
@@ -127,7 +127,7 @@ class ResponseFormat(OpenAIBaseModel):

 class StreamOptions(OpenAIBaseModel):
    include_usage: Optional[bool] = True
-    continuous_usage_stats: Optional[bool] = True
+    continuous_usage_stats: Optional[bool] = False


 class FunctionDefinition(OpenAIBaseModel):