Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
1f5d178e
Unverified
Commit
1f5d178e
authored
Jun 26, 2025
by
Michael Goin
Committed by
GitHub
Jun 26, 2025
Browse files
Revert "[Bugfix] default set cuda_graph_sizes to max_num_seqs for v1 engine" (#20128)
parent
27c065df
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
9 deletions
+4
-9
vllm/config.py
vllm/config.py
+4
-9
No files found.
vllm/config.py
View file @
1f5d178e
...
...
@@ -2042,12 +2042,11 @@ class SchedulerConfig:
NOTE: This will be replaced by speculative config in the future; it is
present to enable correctness tests until then."""
cuda_graph_sizes
:
list
[
int
]
=
field
(
default_factory
=
list
)
"""Cuda graph capture sizes
1. if none provided, then default set to [max_num_seqs]
2. if one value is provided, then the capture list would follow the
cuda_graph_sizes
:
list
[
int
]
=
field
(
default_factory
=
lambda
:
[
512
])
"""Cuda graph capture sizes, default is 512.
1. if one value is provided, then the capture list would follow the
pattern: [1, 2, 4] + [i for i in range(8, cuda_graph_sizes + 1, 8)]
3
. more than one value (e.g. 1 2 128) is provided, then the capture list
2
. more than one value (e.g. 1 2 128) is provided, then the capture list
will follow the provided list."""
delay_factor
:
float
=
0.0
...
...
@@ -2212,10 +2211,6 @@ class SchedulerConfig:
self
.
max_num_partial_prefills
,
self
.
max_long_partial_prefills
,
self
.
long_prefill_token_threshold
)
# If cuda_graph_sizes is not specified, default set to [max_num_seqs].
if
not
self
.
cuda_graph_sizes
:
self
.
cuda_graph_sizes
=
[
self
.
max_num_seqs
]
@
model_validator
(
mode
=
'after'
)
def
_verify_args
(
self
)
->
Self
:
if
(
self
.
max_num_batched_tokens
<
self
.
max_model_len
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment