Commit 424155cd authored by Yuexin Wu's avatar Yuexin Wu Committed by A. Unique TensorFlower
Browse files

Change the default value of `share_rezero` to be False.

PiperOrigin-RevId: 465482790
parent 891b9a9f
......@@ -253,7 +253,7 @@ class FunnelTransformerEncoder(tf.keras.layers.Layer):
norm_first: bool = False,
transformer_cls: Union[
str, tf.keras.layers.Layer] = layers.TransformerEncoderBlock,
share_rezero: bool = True,
share_rezero: bool = False,
**kwargs):
super().__init__(**kwargs)
activation = tf.keras.activations.get(inner_activation)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment