Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
8423aef4
Unverified
Commit
8423aef4
authored
Aug 31, 2024
by
Robert Shaw
Committed by
GitHub
Aug 31, 2024
Browse files
[BugFix][Core] Multistep Fix Crash on Request Cancellation (#8059)
parent
4f5d8446
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
7 additions
and
1 deletion
+7
-1
vllm/engine/output_processor/multi_step.py
vllm/engine/output_processor/multi_step.py
+7
-1
No files found.
vllm/engine/output_processor/multi_step.py
View file @
8423aef4
...
@@ -88,9 +88,15 @@ class MultiStepOutputProcessor(SequenceGroupOutputProcessor):
...
@@ -88,9 +88,15 @@ class MultiStepOutputProcessor(SequenceGroupOutputProcessor):
# TODO: Add support for async if necessary
# TODO: Add support for async if necessary
assert
not
is_async
assert
not
is_async
# Sequences can be in RUNNING or FINISHED_ABORTED state
# once scheduled, as a sequence is moved to FINSIHED_ABORTED
# if a client disconnects from the api server.
seqs
=
sequence_group
.
get_seqs
(
status
=
SequenceStatus
.
RUNNING
)
seqs
=
sequence_group
.
get_seqs
(
status
=
SequenceStatus
.
RUNNING
)
if
seqs
is
None
:
seqs
=
sequence_group
.
get_seqs
(
status
=
SequenceStatus
.
FINISHED_ABORTED
)
assert
seqs
,
"
e
xpected
running
sequences"
assert
seqs
,
"
E
xpected
RUNNING or FINISHED_ABORTED
sequences"
assert
len
(
seqs
)
==
1
,
(
assert
len
(
seqs
)
==
1
,
(
"Beam search not supported in multi-step decoding."
)
"Beam search not supported in multi-step decoding."
)
seq
=
seqs
[
0
]
seq
=
seqs
[
0
]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment