Skip to content

GitLab

Unverified Commit 1ae81903 authored by Jingyue Wu's avatar Jingyue Wu Committed by GitHub
Browse files

Fix an invalid reference in the doc (#1362)

parent e7bfc0c5
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
examples/pytorch/comm_gemm_overlap/README.md
View file @ 1ae81903
......@@ -16,7 +16,7 @@
Forward and backward passes with layer weights distributed over all GPUs in a single node.
```bash
$ torchrun --nnodes=1 --nproc-per-node=$(nvidia-smi -L | wc -l) ln_mlp_with_overlap.py
$ torchrun --nnodes=1 --nproc-per-node=$(nvidia-smi -L | wc -l) te_layer_with_overlap.py
# Sample output on 8x H100s:
# [rank0:node0] |-- Created tensor-parallel group: [0, 1, 2, 3, 4, 5, 6, 7]
......@@ -70,7 +70,7 @@ Uses `torch.nn.parallel.DistributedDataParallel` for replicatin the model across
groups in a single node.
```bash
$ torchrun --nnodes=1 --nproc-per-node=$(nvidia-smi -L | wc -l) ln_mlp_overlap.py --num-replicas 2
$ torchrun --nnodes=1 --nproc-per-node=$(nvidia-smi -L | wc -l) te_layer_with_overlap.py --num-replicas 2
# Sample output on 8x H100s:
# [rank0:node0] |-- Created tensor-parallel group: [0, 1, 2, 3]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment