Unverified Commit 8278825b authored by Mateusz Sokół's avatar Mateusz Sokół Committed by GitHub
Browse files

DOC: TPU mention fix (#38129)


Signed-off-by: default avatarMateusz Sokół <mat646@gmail.com>
parent acf7292b
......@@ -214,14 +214,18 @@ class ParallelConfig:
distributed_executor_backend: (
str | DistributedExecutorBackend | type[Executor] | None
) = None
"""Backend to use for distributed model workers, either "ray" or "mp"
"""
Backend to use for distributed model workers, either "ray" or "mp"
(multiprocessing). If the product of pipeline_parallel_size and tensor_parallel_size
is less than or equal to the number of GPUs available, "mp" will be used to
keep processing on a single host. Otherwise, an error will be raised. To use "mp"
you must also set nnodes, and to use "ray" you must manually set
distributed_executor_backend to "ray".
Note that tpu only support Ray for distributed inference."""
Note:
[TPU](https://docs.vllm.ai/projects/tpu/en/latest/) platform only supports Ray
for distributed inference.
"""
worker_cls: str = "auto"
"""The full name of the worker class to use. If "auto", the worker class
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment