Commit 77452fe3 authored by Yuexin Wu's avatar Yuexin Wu Committed by A. Unique TensorFlower
Browse files

Change the default value of `share_rezero` to be False.

PiperOrigin-RevId: 465482790
parent d0b623d4
...@@ -253,7 +253,7 @@ class FunnelTransformerEncoder(tf.keras.layers.Layer): ...@@ -253,7 +253,7 @@ class FunnelTransformerEncoder(tf.keras.layers.Layer):
norm_first: bool = False, norm_first: bool = False,
transformer_cls: Union[ transformer_cls: Union[
str, tf.keras.layers.Layer] = layers.TransformerEncoderBlock, str, tf.keras.layers.Layer] = layers.TransformerEncoderBlock,
share_rezero: bool = True, share_rezero: bool = False,
**kwargs): **kwargs):
super().__init__(**kwargs) super().__init__(**kwargs)
activation = tf.keras.activations.get(inner_activation) activation = tf.keras.activations.get(inner_activation)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment