Skip to content

GitLab

Commit e1f1aa06 authored by Mostofa Patwary's avatar Mostofa Patwary
Browse files

removed self_attention_pos_emb

parent 5b7736ca
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 1 deletion
megatron/model/transformer.py
View file @ e1f1aa06
...@@ -741,7 +741,6 @@ class ParallelTransformerLayer(MegatronModule): ...@@ -741,7 +741,6 @@ class ParallelTransformerLayer(MegatronModule):
# Layer norm at the beginning of the transformer layer. # Layer norm at the beginning of the transformer layer.
layernorm_output = self.input_layernorm(hidden_states) layernorm_output = self.input_layernorm(hidden_states)
# Self attention. # Self attention.
self_attention_pos_emb = None
attention_output, attention_bias = \ attention_output, attention_bias = \
self.self_attention( self.self_attention(
layernorm_output, layernorm_output,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment