Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
91f4522c
Unverified
Commit
91f4522c
authored
Aug 21, 2024
by
William Lin
Committed by
GitHub
Aug 21, 2024
Browse files
[multi-step] Raise error if not using async engine (#7703)
parent
1b32e026
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
0 deletions
+5
-0
vllm/engine/llm_engine.py
vllm/engine/llm_engine.py
+5
-0
No files found.
vllm/engine/llm_engine.py
View file @
91f4522c
...
@@ -1302,6 +1302,11 @@ class LLMEngine:
...
@@ -1302,6 +1302,11 @@ class LLMEngine:
raise
NotImplementedError
(
raise
NotImplementedError
(
"Pipeline parallelism is only supported through AsyncLLMEngine "
"Pipeline parallelism is only supported through AsyncLLMEngine "
"as performance will be severely degraded otherwise."
)
"as performance will be severely degraded otherwise."
)
if
self
.
scheduler_config
.
num_scheduler_steps
>
1
:
raise
NotImplementedError
(
"Multiple scheduler steps (multi-step) are only supported "
"through AsyncLLMEngine. "
)
seq_group_metadata_list
,
scheduler_outputs
=
self
.
scheduler
[
seq_group_metadata_list
,
scheduler_outputs
=
self
.
scheduler
[
0
].
schedule
()
0
].
schedule
()
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment