Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
40766ca1
Unverified
Commit
40766ca1
authored
Dec 11, 2024
by
Rafael Vasquez
Committed by
GitHub
Dec 11, 2024
Browse files
[Bugfix]: Clamp `-inf` logprob values in prompt_logprobs (#11073)
Signed-off-by:
Rafael Vasquez
<
rafvasq21@gmail.com
>
parent
2e32f5d2
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
0 deletions
+6
-0
vllm/entrypoints/openai/serving_completion.py
vllm/entrypoints/openai/serving_completion.py
+6
-0
No files found.
vllm/entrypoints/openai/serving_completion.py
View file @
40766ca1
...
@@ -392,6 +392,12 @@ class OpenAIServingCompletion(OpenAIServing):
...
@@ -392,6 +392,12 @@ class OpenAIServingCompletion(OpenAIServing):
prompt_token_ids
=
final_res
.
prompt_token_ids
prompt_token_ids
=
final_res
.
prompt_token_ids
assert
prompt_token_ids
is
not
None
assert
prompt_token_ids
is
not
None
prompt_logprobs
=
final_res
.
prompt_logprobs
prompt_logprobs
=
final_res
.
prompt_logprobs
if
prompt_logprobs
:
for
logprob_dict
in
prompt_logprobs
:
if
logprob_dict
:
for
logprob_values
in
logprob_dict
.
values
():
if
logprob_values
.
logprob
==
float
(
'-inf'
):
logprob_values
.
logprob
=
-
9999.0
prompt_text
=
final_res
.
prompt
prompt_text
=
final_res
.
prompt
token_ids
:
GenericSequence
[
int
]
token_ids
:
GenericSequence
[
int
]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment