Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
97a6be95
Unverified
Commit
97a6be95
authored
Aug 12, 2024
by
Andrew Wang
Committed by
GitHub
Aug 13, 2024
Browse files
[Misc] improve logits processors logging message (#7435)
parent
9ba85bc1
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
1 addition
and
2 deletions
+1
-2
tests/entrypoints/openai/test_serving_chat.py
tests/entrypoints/openai/test_serving_chat.py
+0
-1
vllm/entrypoints/openai/logits_processors.py
vllm/entrypoints/openai/logits_processors.py
+1
-1
No files found.
tests/entrypoints/openai/test_serving_chat.py
View file @
97a6be95
...
...
@@ -73,7 +73,6 @@ def test_serving_chat_should_set_correct_max_tokens():
with
suppress
(
Exception
):
asyncio
.
run
(
serving_chat
.
create_chat_completion
(
req
))
# AsyncLLMEngine.generate(inputs, sampling_params, ...)
assert
mock_engine
.
generate
.
call_args
.
args
[
1
].
max_tokens
==
93
req
.
max_tokens
=
10
...
...
vllm/entrypoints/openai/logits_processors.py
View file @
97a6be95
...
...
@@ -71,7 +71,7 @@ def get_logits_processors(
# Check if token_id is within the vocab size
for
token_id
,
bias
in
clamped_logit_bias
.
items
():
if
token_id
<
0
or
token_id
>=
tokenizer
.
vocab_size
:
raise
ValueError
(
"token_id in logit_bias contains "
raise
ValueError
(
f
"token_id
{
token_id
}
in logit_bias contains "
"out-of-vocab token id"
)
logits_processors
.
append
(
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment