"git@developer.sourcefind.cn:OpenDAS/megatron-lm.git" did not exist on "b8940b96ea799cfbc4209e21f73ea6e51ee5b430"
Commit 2ef470e8 authored by mshoeybi's avatar mshoeybi
Browse files

Merge branch 'update-sample-bert-pretrain-args' of...

Merge branch 'update-sample-bert-pretrain-args' of https://github.com/roclark/Megatron-LM into github_small_prs
parents 2c26bf78 3f57f46a
...@@ -23,6 +23,7 @@ python -m torch.distributed.launch $DISTRIBUTED_ARGS \ ...@@ -23,6 +23,7 @@ python -m torch.distributed.launch $DISTRIBUTED_ARGS \
--num-attention-heads 16 \ --num-attention-heads 16 \
--micro-batch-size 2 \ --micro-batch-size 2 \
--global-batch-size 16 \ --global-batch-size 16 \
--seq-length 512 \
--max-position-embeddings 512 \ --max-position-embeddings 512 \
--train-iters 1000000 \ --train-iters 1000000 \
--save $CHECKPOINT_PATH \ --save $CHECKPOINT_PATH \
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment