Skip to content

GitLab

Unverified Commit 2a719e08 authored by Wentao Ye's avatar Wentao Ye Committed by GitHub
Browse files

[Perf] Optimize requests abort (#32211) 


Signed-off-by: default avataryewentao256 <zhyanwentao@126.com>
parent f243abc9
Show whitespace changes
Inline Side-by-side
Showing with 4 additions and 3 deletions
vllm/v1/engine/async_llm.py
View file @ 2a719e08
...@@ -525,6 +525,7 @@ class AsyncLLM(EngineClient): ...@@ -525,6 +525,7 @@ class AsyncLLM(EngineClient):
await asyncio.sleep(0) await asyncio.sleep(0)
# 3) Abort any reqs that finished due to stop strings. # 3) Abort any reqs that finished due to stop strings.
if processed_outputs.reqs_to_abort:
await engine_core.abort_requests_async( await engine_core.abort_requests_async(
processed_outputs.reqs_to_abort processed_outputs.reqs_to_abort
) )
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment