Skip to content

GitLab

"git@developer.sourcefind.cn:OpenDAS/torch-scatter.git" did not exist on "1545db53a61bbd5cd33806d78fc3e96725c3798f"
Commit c187e404 authored by Chen Chen's avatar Chen Chen Committed by A. Unique TensorFlower
Browse files

Internal Change 

PiperOrigin-RevId: 317123221
parent 20f31bf6
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
official/nlp/configs/encoders.py
View file @ c187e404
...@@ -34,7 +34,7 @@ class TransformerEncoderConfig(base_config.Config): ...@@ -34,7 +34,7 @@ class TransformerEncoderConfig(base_config.Config):
num_layers: int = 12 num_layers: int = 12
num_attention_heads: int = 12 num_attention_heads: int = 12
hidden_activation: str = "gelu" hidden_activation: str = "gelu"
intermediate_size: int = 3076 intermediate_size: int = 3072
dropout_rate: float = 0.1 dropout_rate: float = 0.1
attention_dropout_rate: float = 0.1 attention_dropout_rate: float = 0.1
max_position_embeddings: int = 512 max_position_embeddings: int = 512
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment