Unverified Commit e39400a4 authored by Maximilien de Bayser's avatar Maximilien de Bayser Committed by GitHub
Browse files

Fix streaming for granite tool call when <|tool_call|> is present (#11069)


Signed-off-by: default avatarMax de Bayser <mbayser@br.ibm.com>
parent ffa48c91
...@@ -88,7 +88,11 @@ class GraniteToolParser(ToolParser): ...@@ -88,7 +88,11 @@ class GraniteToolParser(ToolParser):
) -> Union[DeltaMessage, None]: ) -> Union[DeltaMessage, None]:
start_idx = consume_space(0, current_text) start_idx = consume_space(0, current_text)
if not current_text or current_text[start_idx] != '[': if current_text[start_idx:].startswith(self.bot_token):
start_idx = consume_space(start_idx + len(self.bot_token),
current_text)
if not current_text or start_idx >= len(current_text)\
or current_text[start_idx] != '[':
return DeltaMessage(content=delta_text) return DeltaMessage(content=delta_text)
# bit mask flags for partial JSON parsing. If the name hasn't been # bit mask flags for partial JSON parsing. If the name hasn't been
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment