Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
norm
vllm
Commits
95592fa0
Unverified
Commit
95592fa0
authored
Sep 19, 2023
by
Roy
Committed by
GitHub
Sep 18, 2023
Browse files
align llm_engine and async_engine. (#1081)
parent
fbe66e1d
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
5 deletions
+4
-5
vllm/engine/async_llm_engine.py
vllm/engine/async_llm_engine.py
+4
-5
No files found.
vllm/engine/async_llm_engine.py
View file @
95592fa0
...
...
@@ -183,10 +183,9 @@ class _AsyncLLMEngine(LLMEngine):
and updates the scheduler with the model outputs. Finally, it decodes
the sequences and returns the newly generated results.
"""
(
seq_group_metadata_list
,
scheduler_outputs
,
early_return
)
=
self
.
_schedule
()
if
early_return
is
not
None
:
return
early_return
seq_group_metadata_list
,
scheduler_outputs
,
ignored
=
self
.
_schedule
()
if
scheduler_outputs
.
is_empty
():
return
ignored
# Execute the model.
output
=
await
self
.
_run_workers_async
(
...
...
@@ -197,7 +196,7 @@ class _AsyncLLMEngine(LLMEngine):
blocks_to_copy
=
scheduler_outputs
.
blocks_to_copy
,
)
return
self
.
_process_model_outputs
(
output
,
scheduler_outputs
)
return
self
.
_process_model_outputs
(
output
,
scheduler_outputs
)
+
ignored
async
def
_run_workers_async
(
self
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment