[Model] Fixed stream generator for gpt-oss + spec-decoding (#26027)

Signed-off-by: Aleksandr Samarin <astrlrd@nebius.com>

[Model] Fixed stream generator for gpt-oss + spec-decoding (#26027)
Signed-off-by: Aleksandr Samarin <astrlrd@nebius.com>
73a99cc2 · Aleksandr Samarin · GitHub · adae0c1f · 73a99cc2
Unverified Commit 73a99cc2 authored Oct 03, 2025 by Aleksandr Samarin Committed by GitHub Oct 03, 2025
Hide whitespace changes
Inline Side-by-side

Showing with 3 additions and 1 deletion

vllm/entrypoints/openai/serving_chat.py vllm/entrypoints/openai/serving_chat.py +3 -1

No files found.
--- a/vllm/entrypoints/openai/serving_chat.py
+++ b/vllm/entrypoints/openai/serving_chat.py
@@ -691,11 +691,13 @@ class OpenAIServingChat(OpenAIServing):
                    if self.use_harmony:
                        harmony_parser = harmony_parsers[i]
                        prev_recipient = harmony_parser.current_recipient
+                        delta_text = ""
                        for token_id in output.token_ids:
                            harmony_parser.process(token_id)
+                            delta_text += (harmony_parser.last_content_delta
+                                           or "")
                        cur_channel = harmony_parser.current_channel
                        cur_recipient = harmony_parser.current_recipient
-                        delta_text = harmony_parser.last_content_delta or ""
                    else:
                        delta_text = output.text