Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
03fb8e79
Unverified
Commit
03fb8e79
authored
Sep 24, 2020
by
Daquan Lin
Committed by
GitHub
Sep 24, 2020
Browse files
Update modeling_tf_longformer.py (#7359)
correct a very small mistake
parent
1ff5bd38
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
src/transformers/modeling_tf_longformer.py
src/transformers/modeling_tf_longformer.py
+1
-1
No files found.
src/transformers/modeling_tf_longformer.py
View file @
03fb8e79
...
...
@@ -348,7 +348,7 @@ class TFLongformerSelfAttention(tf.keras.layers.Layer):
# matrix multipication
# bcxd: batch_size * num_heads x chunks x 2window_overlap x head_dim
# bcyd: batch_size * num_heads x chunks x 2window_overlap x head_dim
# bcxy: batch_size * num_heads x chunks x 2window_overlap x window_overlap
# bcxy: batch_size * num_heads x chunks x 2window_overlap x
2
window_overlap
chunked_attention_scores
=
tf
.
einsum
(
"bcxd,bcyd->bcxy"
,
chunked_query
,
chunked_key
)
# multiply
# convert diagonals into columns
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment