Skip to content

GitLab

Commit 2ccacfd1 authored by Jialu Liu's avatar Jialu Liu Committed by A. Unique TensorFlower
Browse files

Internal change 

PiperOrigin-RevId: 404261191
parent 96a662d0
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 1 deletion
official/nlp/projects/teams/experiments/base/wiki_books_pretrain.yaml
View file @ 2ccacfd1
...@@ -27,7 +27,7 @@ task: ...@@ -27,7 +27,7 @@ task:
intermediate_size: 3072 intermediate_size: 3072
max_position_embeddings: 512 max_position_embeddings: 512
num_attention_heads: 12 num_attention_heads: 12
num_layers: 6 num_layers: 12
type_vocab_size: 2 type_vocab_size: 2
vocab_size: 30522 vocab_size: 30522
train_data: train_data:
...@@ -39,6 +39,7 @@ task: ...@@ -39,6 +39,7 @@ task:
seq_length: 512 seq_length: 512
use_next_sentence_label: false use_next_sentence_label: false
use_position_id: false use_position_id: false
cycle_length: 8
validation_data: validation_data:
drop_remainder: true drop_remainder: true
global_batch_size: 256 global_batch_size: 256
......
official/nlp/projects/teams/experiments/small/wiki_books_pretrain.yaml
View file @ 2ccacfd1
...@@ -39,6 +39,7 @@ task: ...@@ -39,6 +39,7 @@ task:
seq_length: 512 seq_length: 512
use_next_sentence_label: false use_next_sentence_label: false
use_position_id: false use_position_id: false
cycle_length: 8
validation_data: validation_data:
drop_remainder: true drop_remainder: true
global_batch_size: 256 global_batch_size: 256
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment