Skip to content

GitLab

Commit f04abf1f authored by zhaoying1's avatar zhaoying1
Browse files

Update run-13b-single.sh

parent 296c1ee2
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
multi_node/run-13b-single.sh
View file @ f04abf1f
...@@ -24,7 +24,7 @@ APP="python3 ../pretrain.py --deepspeed --deepspeed_config ../models/deepspeed_z ...@@ -24,7 +24,7 @@ APP="python3 ../pretrain.py --deepspeed --deepspeed_config ../models/deepspeed_z
--dataset_path $DATASET_PATH --spm_model_path $SPM_MODEL_PATH \ --dataset_path $DATASET_PATH --spm_model_path $SPM_MODEL_PATH \
--config_path ../models/llama/7b_config.json \ --config_path ../models/llama/7b_config.json \
--output_model_path output/7b/ --deepspeed_checkpoint_activations \ --output_model_path output/7b/ --deepspeed_checkpoint_activations \
--world_size ${2} --data_processor alpaca --prefix_lm_loss \ --world_size ${1} --data_processor alpaca --prefix_lm_loss \
--total_steps 10000 --save_checkpoint_steps 500 --batch_size 2 --enable_zero3 \ --total_steps 10000 --save_checkpoint_steps 500 --batch_size 2 --enable_zero3 \
" "
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment