Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
5394ad73
Unverified
Commit
5394ad73
authored
May 05, 2025
by
Chauncey
Committed by
GitHub
May 04, 2025
Browse files
[Bugfix] fix KeyError on top logprobs are special tokens (#17637)
Signed-off-by:
chaunceyjiang
<
chaunceyjiang@gmail.com
>
parent
68e1ee00
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
1 deletion
+2
-1
vllm/entrypoints/openai/serving_chat.py
vllm/entrypoints/openai/serving_chat.py
+2
-1
No files found.
vllm/entrypoints/openai/serving_chat.py
View file @
5394ad73
...
@@ -1111,7 +1111,8 @@ class OpenAIServingChat(OpenAIServing):
...
@@ -1111,7 +1111,8 @@ class OpenAIServingChat(OpenAIServing):
return_as_token_id
is
not
None
else
self
.
return_tokens_as_token_ids
return_as_token_id
is
not
None
else
self
.
return_tokens_as_token_ids
for
i
,
token_id
in
enumerate
(
token_ids
):
for
i
,
token_id
in
enumerate
(
token_ids
):
step_top_logprobs
=
top_logprobs
[
i
]
step_top_logprobs
=
top_logprobs
[
i
]
if
step_top_logprobs
is
None
:
if
step_top_logprobs
is
None
or
step_top_logprobs
.
get
(
token_id
)
is
None
:
token
=
tokenizer
.
decode
(
token_id
)
token
=
tokenizer
.
decode
(
token_id
)
if
should_return_as_token_id
:
if
should_return_as_token_id
:
token
=
f
"token_id:
{
token_id
}
"
token
=
f
"token_id:
{
token_id
}
"
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment