Skip to content

GitLab

Commit b7bb52f0 authored by Marc van Zee's avatar Marc van Zee Committed by A. Unique TensorFlower
Browse files

Internal change 

PiperOrigin-RevId: 425409937
parent 37c3e5ff
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
official/projects/vit/modeling/vit.py
View file @ b7bb52f0
...@@ -145,7 +145,7 @@ class Encoder(tf.keras.layers.Layer): ...@@ -145,7 +145,7 @@ class Encoder(tf.keras.layers.Layer):
self._encoder_layers = [] self._encoder_layers = []
# Set layer norm epsilons to 1e-6 to be consistent with JAX implementation. # Set layer norm epsilons to 1e-6 to be consistent with JAX implementation.
# https://flax.readthedocs.io/en/latest/_autosummary/flax.nn.LayerNorm.html # https://flax.readthedocs.io/en/latest/_autosummary/flax.deprecated.nn.LayerNorm.html
for i in range(self._num_layers): for i in range(self._num_layers):
encoder_layer = nn_blocks.TransformerEncoderBlock( encoder_layer = nn_blocks.TransformerEncoderBlock(
inner_activation=activations.gelu, inner_activation=activations.gelu,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment