Skip to content

GitLab

Commit 1b5b2fe4 authored by A. Unique TensorFlower's avatar A. Unique TensorFlower
Browse files

Minor change in EncoderScaffold 

PiperOrigin-RevId: 327160430
parent 55b5100e
Show whitespace changes
Inline Side-by-side
Showing with 7 additions and 7 deletions
official/nlp/modeling/networks/encoder_scaffold.py
View file @ 1b5b2fe4
...@@ -152,13 +152,13 @@ class EncoderScaffold(tf.keras.Model): ...@@ -152,13 +152,13 @@ class EncoderScaffold(tf.keras.Model):
name='position_embedding') name='position_embedding')
position_embeddings = self._position_embedding_layer(word_embeddings) position_embeddings = self._position_embedding_layer(word_embeddings)
type_embeddings = ( self._type_embedding_layer = layers.OnDeviceEmbedding(
layers.OnDeviceEmbedding(
vocab_size=embedding_cfg['type_vocab_size'], vocab_size=embedding_cfg['type_vocab_size'],
embedding_width=embedding_cfg['hidden_size'], embedding_width=embedding_cfg['hidden_size'],
initializer=embedding_cfg['initializer'], initializer=embedding_cfg['initializer'],
use_one_hot=True, use_one_hot=True,
name='type_embeddings')(type_ids)) name='type_embeddings')
type_embeddings = self._type_embedding_layer(type_ids)
embeddings = tf.keras.layers.Add()( embeddings = tf.keras.layers.Add()(
[word_embeddings, position_embeddings, type_embeddings]) [word_embeddings, position_embeddings, type_embeddings])
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment