Skip to content

GitLab

Commit 3d415d0e authored by Zihang Dai's avatar Zihang Dai
Browse files

fix tpu lm1b

parent b2cb2443
Hide whitespace changes
Inline Side-by-side
Showing with 5 additions and 3 deletions
tf/scripts/lm1b_large_tpu.sh
View file @ 3d415d0e
...@@ -25,7 +25,9 @@ D_INNER=8192 ...@@ -25,7 +25,9 @@ D_INNER=8192
TGT_LEN=32 TGT_LEN=32
MEM_LEN=32 MEM_LEN=32
TRAIN_BSZ=512 TRAIN_BSZ=512
VLIDA_BSZ=512 VALID_BSZ=512
TRAIN_BSZ_PER_HOST=$((TRAIN_BSZ / NUM_HOST))
VALID_BSZ_PER_HOST=$((VALID_BSZ / NUM_HOST))
# Testing # Testing
TEST_TGT_LEN=32 TEST_TGT_LEN=32
...@@ -38,8 +40,8 @@ if [[ $1 == 'train_data' ]]; then ...@@ -38,8 +40,8 @@ if [[ $1 == 'train_data' ]]; then
--data_dir=${LOCAL_DIR}/ \ --data_dir=${LOCAL_DIR}/ \
--dataset=lm1b \ --dataset=lm1b \
--tgt_len=${TGT_LEN} \ --tgt_len=${TGT_LEN} \
--per_host_train_bsz=${TRAIN_BSZ} \ --per_host_train_bsz=${TRAIN_BSZ_PER_HOST} \
--per_host_valid_bsz=${VLIDA_BSZ} \ --per_host_valid_bsz=${VALID_BSZ_PER_HOST} \
--num_core_per_host=${NUM_CORE} \ --num_core_per_host=${NUM_CORE} \
--num_passes=10 \ --num_passes=10 \
--use_tpu=True \ --use_tpu=True \
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment