"vscode:/vscode.git/clone" did not exist on "6fa718a46007ae97e98a5bb3bcfb506748a2473e"
Unverified Commit 8278825b authored by Mateusz Sokół's avatar Mateusz Sokół Committed by GitHub
Browse files

DOC: TPU mention fix (#38129)


Signed-off-by: default avatarMateusz Sokół <mat646@gmail.com>
parent acf7292b
...@@ -214,14 +214,18 @@ class ParallelConfig: ...@@ -214,14 +214,18 @@ class ParallelConfig:
distributed_executor_backend: ( distributed_executor_backend: (
str | DistributedExecutorBackend | type[Executor] | None str | DistributedExecutorBackend | type[Executor] | None
) = None ) = None
"""Backend to use for distributed model workers, either "ray" or "mp" """
Backend to use for distributed model workers, either "ray" or "mp"
(multiprocessing). If the product of pipeline_parallel_size and tensor_parallel_size (multiprocessing). If the product of pipeline_parallel_size and tensor_parallel_size
is less than or equal to the number of GPUs available, "mp" will be used to is less than or equal to the number of GPUs available, "mp" will be used to
keep processing on a single host. Otherwise, an error will be raised. To use "mp" keep processing on a single host. Otherwise, an error will be raised. To use "mp"
you must also set nnodes, and to use "ray" you must manually set you must also set nnodes, and to use "ray" you must manually set
distributed_executor_backend to "ray". distributed_executor_backend to "ray".
Note that tpu only support Ray for distributed inference.""" Note:
[TPU](https://docs.vllm.ai/projects/tpu/en/latest/) platform only supports Ray
for distributed inference.
"""
worker_cls: str = "auto" worker_cls: str = "auto"
"""The full name of the worker class to use. If "auto", the worker class """The full name of the worker class to use. If "auto", the worker class
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment