Skip to content

GitLab

"examples/vscode:/vscode.git/clone" did not exist on "ba7b7a2fe648e0df14909228a56f8c8292d25a0b"
Commit 2ccacfd1 authored by Jialu Liu's avatar Jialu Liu Committed by A. Unique TensorFlower
Browse files

Internal change 

PiperOrigin-RevId: 404261191
parent 96a662d0
Show whitespace changes
Inline Side-by-side
Showing with 3 additions and 1 deletion
official/nlp/projects/teams/experiments/base/wiki_books_pretrain.yaml
View file @ 2ccacfd1
......@@ -27,7 +27,7 @@ task:
intermediate_size: 3072
max_position_embeddings: 512
num_attention_heads: 12
num_layers: 6
num_layers: 12
type_vocab_size: 2
vocab_size: 30522
train_data:
......@@ -39,6 +39,7 @@ task:
seq_length: 512
use_next_sentence_label: false
use_position_id: false
cycle_length: 8
validation_data:
drop_remainder: true
global_batch_size: 256
......
official/nlp/projects/teams/experiments/small/wiki_books_pretrain.yaml
View file @ 2ccacfd1
......@@ -39,6 +39,7 @@ task:
seq_length: 512
use_next_sentence_label: false
use_position_id: false
cycle_length: 8
validation_data:
drop_remainder: true
global_batch_size: 256
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment