Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
silencealiang
Pai-Megatron-Patch
Commits
9e06ecd3
Commit
9e06ecd3
authored
Mar 19, 2025
by
silencealiang
Browse files
Update train_deepseekv3_671B_4nodes.sh
parent
3041681f
Pipeline
#2558
failed with stages
in 0 seconds
Changes
1
Pipelines
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
5 deletions
+5
-5
examples/deepseek_v3/train_deepseekv3_671B_4nodes.sh
examples/deepseek_v3/train_deepseekv3_671B_4nodes.sh
+5
-5
No files found.
examples/deepseek_v3/train_deepseekv3_671B_4nodes.sh
View file @
9e06ecd3
...
...
@@ -28,12 +28,12 @@ export NCCL_NET_GDR_LEVEL=7
export
NCCL_NET_GDR_READ
=
1
export
RCCL_SDMA_COPY_ENABLE
=
0
export
NCCL_IB_HCA
=
mlx5_2:1,mlx5_3:1,mlx5_4:1,mlx5_5:1,mlx5_6:1,mlx5_7:1,mlx5_8:1,mlx5_9:1
export
NCCL_TOPO_FILE
=
"
/public/home/xingjl/dependency/rccl-tests-0204
/topo-input.xml"
export
NCCL_TOPO_FILE
=
"
.
/topo-input.xml"
# enable BatchLinear
export
GROUPED_GEMM_BatchLinear
=
1
export
MP_PP0_LAYERS
=
2
# 是否使能视实际情况而定
export
LD_LIBRARY_PATH
=
/public/home/xingjl/dependency/rocblas-install-0227/lib:
$LD_LIBRARY_PATH
LOCAL_RANK
=
$OMPI_COMM_WORLD_LOCAL_RANK
RANK
=
$OMPI_COMM_WORLD_RANK
WORLD_SIZE
=
$OMPI_COMM_WORLD_SIZE
...
...
@@ -65,9 +65,9 @@ SFT=false
AC
=
none
OPTIMIZER_OFFLOAD
=
false
SAVE_INTERVAL
=
500
DATASET_PATH
=
./deepseekv3_dataset/
mmap_deepseekv3_datasets_text_document
VALID_DATASET_PATH
=
./deepseekv3_dataset/
mmap_deepseekv3_datasets_text_document
PRETRAIN_CHECKPOINT_PATH
=
./
deepseekv3_dataset
DATASET_PATH
=
"patch to
mmap_deepseekv3_datasets_text_document
"
VALID_DATASET_PATH
=
"patch to
mmap_deepseekv3_datasets_text_document
"
PRETRAIN_CHECKPOINT_PATH
=
"patch to
deepseekv3_dataset
"
# the following two values will not be used when SFT is true
TRAIN_TOKENS
=
100000000
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment