Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
e39400a4
Unverified
Commit
e39400a4
authored
Dec 11, 2024
by
Maximilien de Bayser
Committed by
GitHub
Dec 11, 2024
Browse files
Fix streaming for granite tool call when <|tool_call|> is present (#11069)
Signed-off-by:
Max de Bayser
<
mbayser@br.ibm.com
>
parent
ffa48c91
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
1 deletion
+5
-1
vllm/entrypoints/openai/tool_parsers/granite_tool_parser.py
vllm/entrypoints/openai/tool_parsers/granite_tool_parser.py
+5
-1
No files found.
vllm/entrypoints/openai/tool_parsers/granite_tool_parser.py
View file @
e39400a4
...
@@ -88,7 +88,11 @@ class GraniteToolParser(ToolParser):
...
@@ -88,7 +88,11 @@ class GraniteToolParser(ToolParser):
)
->
Union
[
DeltaMessage
,
None
]:
)
->
Union
[
DeltaMessage
,
None
]:
start_idx
=
consume_space
(
0
,
current_text
)
start_idx
=
consume_space
(
0
,
current_text
)
if
not
current_text
or
current_text
[
start_idx
]
!=
'['
:
if
current_text
[
start_idx
:].
startswith
(
self
.
bot_token
):
start_idx
=
consume_space
(
start_idx
+
len
(
self
.
bot_token
),
current_text
)
if
not
current_text
or
start_idx
>=
len
(
current_text
)
\
or
current_text
[
start_idx
]
!=
'['
:
return
DeltaMessage
(
content
=
delta_text
)
return
DeltaMessage
(
content
=
delta_text
)
# bit mask flags for partial JSON parsing. If the name hasn't been
# bit mask flags for partial JSON parsing. If the name hasn't been
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment