Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
c84b519c
Unverified
Commit
c84b519c
authored
Mar 12, 2026
by
Isotr0py
Committed by
GitHub
Mar 11, 2026
Browse files
[Bugfix] Fix negative max_tokens when input prompt is too long (#36789)
Signed-off-by:
Isotr0py
<
mozf@mail2.sysu.edu.cn
>
parent
741ecf06
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
19 additions
and
0 deletions
+19
-0
tests/entrypoints/test_utils.py
tests/entrypoints/test_utils.py
+14
-0
vllm/entrypoints/utils.py
vllm/entrypoints/utils.py
+5
-0
No files found.
tests/entrypoints/test_utils.py
View file @
c84b519c
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
import
pytest
from
vllm.entrypoints.utils
import
get_max_tokens
,
sanitize_message
...
...
@@ -80,3 +82,15 @@ class TestGetMaxTokens:
default_sampling_params
=
{
"max_tokens"
:
2048
},
)
assert
result
==
512
def
test_input_length_exceeds_max_model_len
(
self
):
with
pytest
.
raises
(
ValueError
,
match
=
"Input length .* exceeds model's maximum context length .*"
,
):
get_max_tokens
(
max_model_len
=
100
,
max_tokens
=
50
,
input_length
=
150
,
default_sampling_params
=
{
"max_tokens"
:
2048
},
)
vllm/entrypoints/utils.py
View file @
c84b519c
...
...
@@ -178,6 +178,11 @@ def get_max_tokens(
default_sampling_params
:
dict
,
override_max_tokens
:
int
|
None
=
None
,
)
->
int
:
if
max_model_len
<
input_length
:
raise
ValueError
(
f
"Input length (
{
input_length
}
) exceeds model's maximum "
f
"context length (
{
max_model_len
}
)."
)
model_max_tokens
=
max_model_len
-
input_length
platform_max_tokens
=
current_platform
.
get_max_output_tokens
(
input_length
)
fallback_max_tokens
=
(
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment