Internal Change

PiperOrigin-RevId: 317123221

Internal Change
PiperOrigin-RevId: 317123221
c187e404 · Chen Chen · A. Unique TensorFlower · 20f31bf6 · c187e404
Commit c187e404 authored Jun 18, 2020 by Chen Chen Committed by A. Unique TensorFlower Jun 18, 2020
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

official/nlp/configs/encoders.py official/nlp/configs/encoders.py +1 -1

No files found.
--- a/official/nlp/configs/encoders.py
+++ b/official/nlp/configs/encoders.py
@@ -34,7 +34,7 @@ class TransformerEncoderConfig(base_config.Config):
  num_layers: int = 12
  num_attention_heads: int = 12
  hidden_activation: str = "gelu"
-  intermediate_size: int = 3076
+  intermediate_size: int = 3072
  dropout_rate: float = 0.1
  attention_dropout_rate: float = 0.1
  max_position_embeddings: int = 512