Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Torchaudio
Commits
3f562547
"docs/git@developer.sourcefind.cn:OpenDAS/pytorch3d.git" did not exist on "0a7c354dc1d2e2a6642ef8fbb28a6ad1da89283e"
Unverified
Commit
3f562547
authored
Oct 05, 2021
by
moto
Committed by
GitHub
Oct 05, 2021
Browse files
Replace dropout with Dropout (#1815)
parent
02def7c4
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
torchaudio/models/wav2vec2/components.py
torchaudio/models/wav2vec2/components.py
+2
-2
No files found.
torchaudio/models/wav2vec2/components.py
View file @
3f562547
...
@@ -225,7 +225,7 @@ class SelfAttention(Module):
...
@@ -225,7 +225,7 @@ class SelfAttention(Module):
self
.
embed_dim
=
embed_dim
self
.
embed_dim
=
embed_dim
self
.
num_heads
=
num_heads
self
.
num_heads
=
num_heads
self
.
dropout
=
dropout
self
.
dropout
=
torch
.
nn
.
Dropout
(
dropout
)
self
.
head_dim
=
head_dim
self
.
head_dim
=
head_dim
self
.
scaling
=
self
.
head_dim
**
-
0.5
self
.
scaling
=
self
.
head_dim
**
-
0.5
...
@@ -273,7 +273,7 @@ class SelfAttention(Module):
...
@@ -273,7 +273,7 @@ class SelfAttention(Module):
weights
+=
attention_mask
weights
+=
attention_mask
weights
=
torch
.
nn
.
functional
.
softmax
(
weights
,
dim
=-
1
)
weights
=
torch
.
nn
.
functional
.
softmax
(
weights
,
dim
=-
1
)
weights
=
torch
.
nn
.
functional
.
dropout
(
weights
,
p
=
self
.
dropout
,
training
=
self
.
training
)
weights
=
self
.
dropout
(
weights
)
output
=
weights
@
v
# B, nH, L, Hd
output
=
weights
@
v
# B, nH, L, Hd
output
=
output
.
transpose
(
2
,
1
).
reshape
(
batch_size
,
length
,
embed_dim
)
output
=
output
.
transpose
(
2
,
1
).
reshape
(
batch_size
,
length
,
embed_dim
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment