Unverified Commit 40766ca1 authored by Rafael Vasquez's avatar Rafael Vasquez Committed by GitHub
Browse files

[Bugfix]: Clamp `-inf` logprob values in prompt_logprobs (#11073)


Signed-off-by: default avatarRafael Vasquez <rafvasq21@gmail.com>
parent 2e32f5d2
...@@ -392,6 +392,12 @@ class OpenAIServingCompletion(OpenAIServing): ...@@ -392,6 +392,12 @@ class OpenAIServingCompletion(OpenAIServing):
prompt_token_ids = final_res.prompt_token_ids prompt_token_ids = final_res.prompt_token_ids
assert prompt_token_ids is not None assert prompt_token_ids is not None
prompt_logprobs = final_res.prompt_logprobs prompt_logprobs = final_res.prompt_logprobs
if prompt_logprobs:
for logprob_dict in prompt_logprobs:
if logprob_dict:
for logprob_values in logprob_dict.values():
if logprob_values.logprob == float('-inf'):
logprob_values.logprob = -9999.0
prompt_text = final_res.prompt prompt_text = final_res.prompt
token_ids: GenericSequence[int] token_ids: GenericSequence[int]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment