Skip to content

GitLab

Commit 77361dae authored by liangjing's avatar liangjing
Browse files

update

parent c9289b90
Pipeline #1870 passed with stage
Hide whitespace changes
Inline Side-by-side
Showing with 8 additions and 6 deletions
scripts/llama2_13b.sh
View file @ 77361dae
......@@ -32,8 +32,9 @@ GPT_MODEL_ARGS=(
)
TRAINING_ARGS=(
--transformer-impl local
--use-legacy-models
--log-throughput
--transformer-impl transformer_engine
--use-mcore-models
--micro-batch-size 1
--global-batch-size 256
--train-iters 100
......
scripts/llama2_70b.sh
View file @ 77361dae
......@@ -36,8 +36,9 @@ GPT_MODEL_ARGS=(
)
TRAINING_ARGS=(
--transformer-impl local
--use-legacy-models
--log-throughput
--transformer-impl transformer_engine
--use-mcore-models
--micro-batch-size 1
--global-batch-size 512
--train-iters 100
......
scripts/llama2_7b.sh
View file @ 77361dae
......@@ -32,8 +32,8 @@ GPT_MODEL_ARGS=(
TRAINING_ARGS=(
--log-throughput
--transformer-impl local
--use-legacy-models
--transformer-impl transformer_engine
--use-mcore-models
--micro-batch-size 1
--global-batch-size 240
--train-iters 100
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment