Commit 2ef470e8 authored by mshoeybi's avatar mshoeybi
Browse files

Merge branch 'update-sample-bert-pretrain-args' of...

Merge branch 'update-sample-bert-pretrain-args' of https://github.com/roclark/Megatron-LM into github_small_prs
parents 2c26bf78 3f57f46a
......@@ -23,6 +23,7 @@ python -m torch.distributed.launch $DISTRIBUTED_ARGS \
--num-attention-heads 16 \
--micro-batch-size 2 \
--global-batch-size 16 \
--seq-length 512 \
--max-position-embeddings 512 \
--train-iters 1000000 \
--save $CHECKPOINT_PATH \
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment