Skip to content

GitLab

Commit 6f016785 authored by silencealiang's avatar silencealiang
Browse files

Update train_gpt_567B_1nodes.sh

parent e2ce7847
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
examples/gpt3/train_gpt_567B_1nodes.sh
View file @ 6f016785
...@@ -124,8 +124,8 @@ HIP_PROFIE_ARGS=( ...@@ -124,8 +124,8 @@ HIP_PROFIE_ARGS=(
MODEL_PARALLEL_ARGS=( MODEL_PARALLEL_ARGS=(
--tensor-model-parallel-size 2 --tensor-model-parallel-size 2
--pipeline-model-parallel-size 1 --pipeline-model-parallel-size 1
--expert-model-parallel-size 8 --expert-model-parallel-size 4
--expert-tensor-parallel-size 1 --expert-tensor-parallel-size 2
--use-distributed-optimizer --use-distributed-optimizer
--sequence-parallel --sequence-parallel
) )
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment