Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
8ea0c275
Unverified
Commit
8ea0c275
authored
Aug 17, 2025
by
Woosuk Kwon
Committed by
GitHub
Aug 17, 2025
Browse files
[Misc] Minor code cleanup for _get_prompt_logprobs_dict (#23064)
Signed-off-by:
Woosuk Kwon
<
woosuk.kwon@berkeley.edu
>
parent
0fc8fa75
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
4 deletions
+3
-4
vllm/v1/worker/gpu_model_runner.py
vllm/v1/worker/gpu_model_runner.py
+3
-4
No files found.
vllm/v1/worker/gpu_model_runner.py
View file @
8ea0c275
...
...
@@ -1722,7 +1722,7 @@ class GPUModelRunner(LoRAModelRunnerMixin, KVConnectorModelRunnerMixin):
# Compute prompt logprobs if needed.
prompt_logprobs_dict
=
self
.
_get_prompt_logprobs_dict
(
hidden_states
[:
num_scheduled_tokens
],
scheduler_output
,
scheduler_output
.
num_scheduled_tokens
,
)
# Get the valid generated tokens.
...
...
@@ -2064,7 +2064,7 @@ class GPUModelRunner(LoRAModelRunnerMixin, KVConnectorModelRunnerMixin):
def
_get_prompt_logprobs_dict
(
self
,
hidden_states
:
torch
.
Tensor
,
schedule
r_output
:
"SchedulerOutput"
,
num_
schedule
d_tokens
:
dict
[
str
,
int
]
,
)
->
dict
[
str
,
Optional
[
LogprobsTensors
]]:
num_prompt_logprobs_dict
=
self
.
input_batch
.
num_prompt_logprobs
if
not
num_prompt_logprobs_dict
:
...
...
@@ -2077,8 +2077,7 @@ class GPUModelRunner(LoRAModelRunnerMixin, KVConnectorModelRunnerMixin):
# maintainable loop over optimal performance.
completed_prefill_reqs
=
[]
for
req_id
,
num_prompt_logprobs
in
num_prompt_logprobs_dict
.
items
():
num_tokens
=
scheduler_output
.
num_scheduled_tokens
[
req_id
]
num_tokens
=
num_scheduled_tokens
[
req_id
]
# Get metadata for this request.
request
=
self
.
requests
[
req_id
]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment