Unverified Commit bd6d5a74 authored by AlonKejzman's avatar AlonKejzman Committed by GitHub
Browse files

[gpt-oss] Fix harmony parser in streaming responses (#30205)


Signed-off-by: default avatarAlonKejzman <alonkeizman@gmail.com>
parent 256a33ec
...@@ -811,6 +811,11 @@ class OpenAIServingChat(OpenAIServing): ...@@ -811,6 +811,11 @@ class OpenAIServingChat(OpenAIServing):
delta_text += harmony_parser.last_content_delta or "" delta_text += harmony_parser.last_content_delta or ""
cur_channel = harmony_parser.current_channel cur_channel = harmony_parser.current_channel
cur_recipient = harmony_parser.current_recipient cur_recipient = harmony_parser.current_recipient
# handle the case where several tokens where generated at once
# including the final token, leading to a delta in the text
# but the current channel to be empty (start state)
if not cur_channel and delta_text:
cur_channel = "final"
else: else:
delta_text = output.text delta_text = output.text
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment