Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
text-generation-inference
Commits
86bca365
Unverified
Commit
86bca365
authored
Apr 21, 2023
by
OlivierDehaene
Committed by
GitHub
Apr 21, 2023
Browse files
fix(server): fix flash causal (#218)
parent
afc5b999
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
1 deletion
+4
-1
server/text_generation_server/models/flash_causal_lm.py
server/text_generation_server/models/flash_causal_lm.py
+4
-1
No files found.
server/text_generation_server/models/flash_causal_lm.py
View file @
86bca365
...
...
@@ -453,7 +453,10 @@ class FlashCausalLM(Model):
)
# Set in batch in case it needs to be used later in concatenate()
batch
.
past_pad
=
self
.
past_pad
if
len
(
batch
)
!=
1
:
if
len
(
batch
)
==
1
:
# present is already pre-padded
batch
.
past_key_values
=
present
else
:
# Add padding after each sequence
# This will have the correct shape after the final past_key_values concatenation before the model
# forward
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment