Unverified Commit 1b491ad7 authored by Jiarui Fang's avatar Jiarui Fang Committed by GitHub
Browse files

[doc] update docstring in ProcessGroup (#1468)

parent b73fb7a0
......@@ -31,7 +31,7 @@ PYTORCHPGDICT_ = PyTorchProcessGroupDict()
class ProcessGroup:
"""ProcessGroup
Process Group contains group partition for Tensor Parallel and Data Parallel.
Process Group indicates how processes are organized in groups for parallel execution using Tensor Parallelism and Data Parallelism.
NOTE, the ProcessGroup must be used after `torch.distributed.initialize()`
......@@ -40,8 +40,8 @@ class ProcessGroup:
rank: the global rank of the current process.
ranks: List[int], a list of rank id belongings to this process group.
backend: str, the backend of the process group.
tp_degree: Optional[int], tensor parallelism degree, default None means 1
dp_degree: Optional[int], data parallelism degree, default None means len(ranks)
tp_degree: Optional[int], tensor parallelism degree. How many processes are inside a tp process group. default None means 1.
dp_degree: Optional[int], data parallelism degree. How many processes are inside a dp process group. . default None means len(ranks).
"""
def __init__(self,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment