Skip to content

GitLab

Commit b01809dd authored by wxj's avatar wxj
Browse files

Update Llama_pretraining.sh

parent 3e02567a
Pipeline #2445 passed with stage
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
Llama_pretraining.sh
View file @ b01809dd
...@@ -12,7 +12,7 @@ export NCCL_P2P_LEVEL=PXB # SYS ...@@ -12,7 +12,7 @@ export NCCL_P2P_LEVEL=PXB # SYS
export NCCL_ALGO=Ring export NCCL_ALGO=Ring
export NCCL_NCHANNELS_PER_PEER=16 export NCCL_NCHANNELS_PER_PEER=16
export NCCL_MIN_NCHANNELS=20 export NCCL_MIN_NCHANNELS=20 # channel
export NCCL_IB_TIMEOUT=22 export NCCL_IB_TIMEOUT=22
export CUDA_DEVICE_MAX_CONNECTIONS=1 export CUDA_DEVICE_MAX_CONNECTIONS=1
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment