Unverified Commit 1a0f4def authored by Wentao Ye's avatar Wentao Ye Committed by GitHub
Browse files

[Log] Add Warning for `LLM(data_parallel_size=k)` single-process DP Usage (#27282)


Signed-off-by: default avataryewentao256 <zhyanwentao@126.com>
parent 843af7f7
...@@ -285,6 +285,16 @@ class LLM: ...@@ -285,6 +285,16 @@ class LLM:
else: else:
structured_outputs_instance = StructuredOutputsConfig() structured_outputs_instance = StructuredOutputsConfig()
# warn about single-process data parallel usage.
_dps = int(kwargs.get("data_parallel_size", 1))
if _dps > 1:
raise ValueError(
f"LLM(data_parallel_size={_dps}) is not supported for single-"
"process usage and may hang. Please use "
"the explicit multi-process data-parallel example at "
"'examples/offline_inference/data_parallel.py'."
)
engine_args = EngineArgs( engine_args = EngineArgs(
model=model, model=model,
runner=runner, runner=runner,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment