Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
df143026
Unverified
Commit
df143026
authored
Mar 22, 2025
by
Andy Lo
Committed by
GitHub
Mar 21, 2025
Browse files
[Bugfix][V0] Multi-sequence logprobs streaming edge case (#15259)
Signed-off-by:
Andy Lo
<
andy@mistral.ai
>
parent
4c69e228
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
1 deletion
+6
-1
vllm/outputs.py
vllm/outputs.py
+6
-1
No files found.
vllm/outputs.py
View file @
df143026
...
@@ -223,7 +223,12 @@ class RequestOutput:
...
@@ -223,7 +223,12 @@ class RequestOutput:
if
delta
:
if
delta
:
# Slice logprobs delta if applicable
# Slice logprobs delta if applicable
if
output_logprobs
:
if
output_logprobs
:
# num_output_tokens can be 0 when n > 1 and request finishes
# before the others
if
num_output_tokens
>
0
:
output_logprobs
=
output_logprobs
[
-
num_output_tokens
:]
output_logprobs
=
output_logprobs
[
-
num_output_tokens
:]
else
:
output_logprobs
=
None
# Don't include prompt if this is after the first output
# Don't include prompt if this is after the first output
# containing decode token ids
# containing decode token ids
if
include_prompt
and
seq
.
get_output_len
()
>
num_output_tokens
:
if
include_prompt
and
seq
.
get_output_len
()
>
num_output_tokens
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment