fix: pass back `usage` when using text mode with VLLM (#5336)

Signed-off-by: Qidong Su <soodoshll@gmail.com>

fix: pass back `usage` when using text mode with VLLM (#5336)
Signed-off-by: Qidong Su <soodoshll@gmail.com>
68235126 · Qidong Su · GitHub · c8770464 · 68235126
Unverified Commit 68235126 authored Jan 12, 2026 by Qidong Su Committed by GitHub Jan 12, 2026
Hide whitespace changes
Inline Side-by-side

Showing with 5 additions and 0 deletions

components/src/dynamo/vllm/handlers.py components/src/dynamo/vllm/handlers.py +5 -0

No files found.
--- a/components/src/dynamo/vllm/handlers.py
+++ b/components/src/dynamo/vllm/handlers.py
@@ -1300,6 +1300,11 @@ class DecodeWorkerHandler(BaseWorkerHandler):
                        "choices": [choice_data],
                    }

+                    if output.finish_reason:
+                        chunk["usage"] = BaseWorkerHandler._build_completion_usage(
+                            request_output=res,
+                        )
+
                    yield chunk

            except EngineDeadError as e: