Skip to content

GitLab

Commit 77361dae authored by liangjing's avatar liangjing
Browse files

update

parent c9289b90
Pipeline #1870 passed with stage
Hide whitespace changes
Inline Side-by-side
Showing with 8 additions and 6 deletions
scripts/llama2_13b.sh
View file @ 77361dae
...@@ -32,8 +32,9 @@ GPT_MODEL_ARGS=( ...@@ -32,8 +32,9 @@ GPT_MODEL_ARGS=(
) )
TRAINING_ARGS=( TRAINING_ARGS=(
--transformer-impl local --log-throughput
--use-legacy-models --transformer-impl transformer_engine
--use-mcore-models
--micro-batch-size 1 --micro-batch-size 1
--global-batch-size 256 --global-batch-size 256
--train-iters 100 --train-iters 100
......
scripts/llama2_70b.sh
View file @ 77361dae
...@@ -36,8 +36,9 @@ GPT_MODEL_ARGS=( ...@@ -36,8 +36,9 @@ GPT_MODEL_ARGS=(
) )
TRAINING_ARGS=( TRAINING_ARGS=(
--transformer-impl local --log-throughput
--use-legacy-models --transformer-impl transformer_engine
--use-mcore-models
--micro-batch-size 1 --micro-batch-size 1
--global-batch-size 512 --global-batch-size 512
--train-iters 100 --train-iters 100
......
scripts/llama2_7b.sh
View file @ 77361dae
...@@ -32,8 +32,8 @@ GPT_MODEL_ARGS=( ...@@ -32,8 +32,8 @@ GPT_MODEL_ARGS=(
TRAINING_ARGS=( TRAINING_ARGS=(
--log-throughput --log-throughput
--transformer-impl local --transformer-impl transformer_engine
--use-legacy-models --use-mcore-models
--micro-batch-size 1 --micro-batch-size 1
--global-batch-size 240 --global-batch-size 240
--train-iters 100 --train-iters 100
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment