Skip to content

GitLab

Unverified Commit 4708f13a authored by Michael Goin's avatar Michael Goin Committed by GitHub
Browse files

[Bugfix] Fix default behavior/fallback for pp in v1 (#16057) 


Signed-off-by: default avatarmgoin <mgoin64@gmail.com>
parent a6d042df
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
vllm/engine/arg_utils.py
View file @ 4708f13a
...@@ -1521,8 +1521,9 @@ class EngineArgs: ...@@ -1521,8 +1521,9 @@ class EngineArgs:
# PP is supported on V1 with Ray distributed executor, # PP is supported on V1 with Ray distributed executor,
# but off for MP distributed executor for now. # but off for MP distributed executor for now.
if (self.pipeline_parallel_size > 1 if (self.pipeline_parallel_size > 1
and self.distributed_executor_backend == "mp" and self.distributed_executor_backend != "ray"):
and _warn_or_fallback("PP (MP distributed executor)")): name = "Pipeline Parallelism without Ray distributed executor"
_raise_or_fallback(feature_name=name, recommend_to_remove=False)
return False return False
# ngram is supported on V1, but off by default for now. # ngram is supported on V1, but off by default for now.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment