Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
b0b77c46
Unverified
Commit
b0b77c46
authored
Dec 18, 2025
by
Nick Hill
Committed by
GitHub
Dec 18, 2025
Browse files
[BugFix] Fix spec decode + structured outputs + preemption edge case (#30916)
Signed-off-by:
Nick Hill
<
nhill@redhat.com
>
parent
634a14bd
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
1 deletion
+5
-1
vllm/v1/worker/gpu_model_runner.py
vllm/v1/worker/gpu_model_runner.py
+5
-1
No files found.
vllm/v1/worker/gpu_model_runner.py
View file @
b0b77c46
...
...
@@ -3393,9 +3393,13 @@ class GPUModelRunner(
return
async_output
def
take_draft_token_ids
(
self
)
->
DraftTokenIds
|
None
:
if
self
.
_draft_token_ids
is
None
:
if
not
self
.
num_spec_tokens
:
return
None
req_ids
=
self
.
input_batch
.
req_ids
if
self
.
_draft_token_ids
is
None
:
return
DraftTokenIds
(
req_ids
,
[[]
for
_
in
req_ids
])
if
isinstance
(
self
.
_draft_token_ids
,
torch
.
Tensor
):
draft_token_ids
=
self
.
_draft_token_ids
.
tolist
()
else
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment