Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
5b2dcbf0
Unverified
Commit
5b2dcbf0
authored
May 09, 2025
by
inkcherry
Committed by
GitHub
May 09, 2025
Browse files
Fix Whisper crash caused by invalid``` max_num_batched_tokens``` config (#17853)
Signed-off-by:
inkcherry
<
mingzhi.liu@intel.com
>
parent
6e4a93e3
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
14 additions
and
0 deletions
+14
-0
vllm/config.py
vllm/config.py
+14
-0
No files found.
vllm/config.py
View file @
5b2dcbf0
...
@@ -2050,6 +2050,13 @@ class SchedulerConfig:
...
@@ -2050,6 +2050,13 @@ class SchedulerConfig:
_MULTIMODAL_MODEL_MAX_NUM_BATCHED_TOKENS
,
_MULTIMODAL_MODEL_MAX_NUM_BATCHED_TOKENS
,
)
)
# When using default settings,
# Ensure max_num_batched_tokens does not exceed model limit.
# Some models (e.g., Whisper) have embeddings tied to max length.
self
.
max_num_batched_tokens
=
min
(
self
.
max_num_seqs
*
self
.
max_model_len
,
self
.
max_num_batched_tokens
)
self
.
max_num_encoder_input_tokens
=
self
.
max_num_batched_tokens
self
.
max_num_encoder_input_tokens
=
self
.
max_num_batched_tokens
self
.
encoder_cache_size
=
self
.
max_num_batched_tokens
self
.
encoder_cache_size
=
self
.
max_num_batched_tokens
...
@@ -2090,6 +2097,13 @@ class SchedulerConfig:
...
@@ -2090,6 +2097,13 @@ class SchedulerConfig:
"be greater than or equal to max_num_seqs "
"be greater than or equal to max_num_seqs "
f
"(
{
self
.
max_num_seqs
}
)."
)
f
"(
{
self
.
max_num_seqs
}
)."
)
if
self
.
max_num_batched_tokens
>
self
.
max_num_seqs
*
self
.
max_model_len
:
logger
.
warning
(
"max_num_batched_tokens (%d) exceeds max_num_seqs"
"* max_model_len (%d). This may lead to unexpected behavior."
,
self
.
max_num_batched_tokens
,
self
.
max_num_seqs
*
self
.
max_model_len
)
if
self
.
num_lookahead_slots
<
0
:
if
self
.
num_lookahead_slots
<
0
:
raise
ValueError
(
raise
ValueError
(
"num_lookahead_slots "
"num_lookahead_slots "
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment