Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
c87ebc3e
Unverified
Commit
c87ebc3e
authored
Jul 01, 2024
by
Nick Hill
Committed by
GitHub
Jul 01, 2024
Browse files
[BugFix] Ensure worker model loop is always stopped at the right time (#5987)
parent
c4059ea5
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
vllm/engine/llm_engine.py
vllm/engine/llm_engine.py
+1
-1
No files found.
vllm/engine/llm_engine.py
View file @
c87ebc3e
...
@@ -838,7 +838,7 @@ class LLMEngine:
...
@@ -838,7 +838,7 @@ class LLMEngine:
# Tracing
# Tracing
self
.
do_tracing
(
scheduler_outputs
)
self
.
do_tracing
(
scheduler_outputs
)
if
not
request_outputs
:
if
not
self
.
has_unfinished_requests
()
:
# Stop the execute model loop in parallel workers until there are
# Stop the execute model loop in parallel workers until there are
# more requests to process. This avoids waiting indefinitely in
# more requests to process. This avoids waiting indefinitely in
# torch.distributed ops which may otherwise timeout, and unblocks
# torch.distributed ops which may otherwise timeout, and unblocks
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment