Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
parler-tts
Commits
e25b8ba0
Commit
e25b8ba0
authored
Feb 23, 2024
by
Yoach Lacombe
Browse files
fix position embedding computation
parent
cb44e483
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
stable_speech/modeling_stable_speech.py
stable_speech/modeling_stable_speech.py
+2
-2
No files found.
stable_speech/modeling_stable_speech.py
View file @
e25b8ba0
...
@@ -136,7 +136,7 @@ class StableSpeechSinusoidalPositionalEmbedding(nn.Module):
...
@@ -136,7 +136,7 @@ class StableSpeechSinusoidalPositionalEmbedding(nn.Module):
@
torch
.
no_grad
()
@
torch
.
no_grad
()
def
forward
(
self
,
input_ids
:
torch
.
Tensor
,
past_key_values_length
:
int
=
0
):
def
forward
(
self
,
input_ids
:
torch
.
Tensor
,
past_key_values_length
:
int
=
0
):
bsz
,
codebooks
,
seq_len
=
input_ids
.
size
()
bsz
,
seq_len
,
_
=
input_ids
.
size
()
# Create the position ids from the input token ids.
# Create the position ids from the input token ids.
position_ids
=
(
torch
.
arange
(
seq_len
)
+
past_key_values_length
).
to
(
input_ids
.
device
)
position_ids
=
(
torch
.
arange
(
seq_len
)
+
past_key_values_length
).
to
(
input_ids
.
device
)
# expand embeddings if needed
# expand embeddings if needed
...
@@ -754,7 +754,7 @@ class StableSpeechDecoder(StableSpeechPreTrainedModel):
...
@@ -754,7 +754,7 @@ class StableSpeechDecoder(StableSpeechPreTrainedModel):
)
)
# embed positions
# embed positions
positions
=
self
.
embed_positions
(
input
,
past_key_values_length
)
positions
=
self
.
embed_positions
(
input
s_embeds
,
past_key_values_length
)
hidden_states
=
inputs_embeds
+
positions
.
to
(
inputs_embeds
.
device
)
hidden_states
=
inputs_embeds
+
positions
.
to
(
inputs_embeds
.
device
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment