Internal change

PiperOrigin-RevId: 397671381

Internal change
PiperOrigin-RevId: 397671381
418903b0 · Hongkun Yu · A. Unique TensorFlower · 615f12cf · 418903b0 · 418903b0
Commit 418903b0 authored Sep 19, 2021 by Hongkun Yu Committed by A. Unique TensorFlower Sep 19, 2021
2 changed files
--- a/official/nlp/configs/models/albert_base.yaml
+++ b/official/nlp/configs/models/albert_base.yaml
+task:
+  model:
+    encoder:
+      type: albert
+      albert:
+        attention_dropout_rate: 0.0
+        dropout_rate: 0.0
+        hidden_activation: gelu
+        hidden_size: 768
+        initializer_range: 0.02
+        intermediate_size: 3072
+        max_position_embeddings: 512
+        num_attention_heads: 12
+        num_layers: 12
+        type_vocab_size: 2
+        vocab_size: 30000
--- a/official/nlp/albert/tf2_albert_encoder_checkpoint_converter.py
+++ b/official/nlp/albert/tf2_albert_encoder_checkpoint_converter.py