Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
2ad6194a
Unverified
Commit
2ad6194a
authored
May 31, 2025
by
Michael Goin
Committed by
GitHub
Jun 01, 2025
Browse files
Let max_num_batched_tokens use human_readable_int for large numbers (#18968)
Signed-off-by:
mgoin
<
mgoin64@gmail.com
>
parent
c594cbf5
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
vllm/engine/arg_utils.py
vllm/engine/arg_utils.py
+1
-1
No files found.
vllm/engine/arg_utils.py
View file @
2ad6194a
...
@@ -224,7 +224,7 @@ def get_kwargs(cls: ConfigType) -> dict[str, Any]:
...
@@ -224,7 +224,7 @@ def get_kwargs(cls: ConfigType) -> dict[str, Any]:
elif
contains_type
(
type_hints
,
int
):
elif
contains_type
(
type_hints
,
int
):
kwargs
[
name
][
"type"
]
=
int
kwargs
[
name
][
"type"
]
=
int
# Special case for large integers
# Special case for large integers
if
name
in
{
"max_model_len"
}:
if
name
in
{
"max_model_len"
,
"max_num_batched_tokens"
}:
kwargs
[
name
][
"type"
]
=
human_readable_int
kwargs
[
name
][
"type"
]
=
human_readable_int
elif
contains_type
(
type_hints
,
float
):
elif
contains_type
(
type_hints
,
float
):
kwargs
[
name
][
"type"
]
=
float
kwargs
[
name
][
"type"
]
=
float
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment