Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
22d33bac
Unverified
Commit
22d33bac
authored
Mar 19, 2025
by
Nick Hill
Committed by
GitHub
Mar 19, 2025
Browse files
[FrontEnd][Perf] `merge_async_iterators` fast-path for single-prompt requests (#15150)
Signed-off-by:
Nick Hill
<
nhill@redhat.com
>
parent
b0e96aae
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
0 deletions
+5
-0
vllm/utils.py
vllm/utils.py
+5
-0
No files found.
vllm/utils.py
View file @
22d33bac
...
@@ -411,6 +411,11 @@ async def merge_async_iterators(
...
@@ -411,6 +411,11 @@ async def merge_async_iterators(
When it yields, it yields a tuple (i, item) where i is the index of the
When it yields, it yields a tuple (i, item) where i is the index of the
iterator that yields the item.
iterator that yields the item.
"""
"""
if
len
(
iterators
)
==
1
:
# Fast-path single iterator case.
async
for
item
in
iterators
[
0
]:
yield
0
,
item
return
loop
=
asyncio
.
get_running_loop
()
loop
=
asyncio
.
get_running_loop
()
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment