Unverified Commit 2ad6194a authored by Michael Goin's avatar Michael Goin Committed by GitHub
Browse files

Let max_num_batched_tokens use human_readable_int for large numbers (#18968)


Signed-off-by: default avatarmgoin <mgoin64@gmail.com>
parent c594cbf5
...@@ -224,7 +224,7 @@ def get_kwargs(cls: ConfigType) -> dict[str, Any]: ...@@ -224,7 +224,7 @@ def get_kwargs(cls: ConfigType) -> dict[str, Any]:
elif contains_type(type_hints, int): elif contains_type(type_hints, int):
kwargs[name]["type"] = int kwargs[name]["type"] = int
# Special case for large integers # Special case for large integers
if name in {"max_model_len"}: if name in {"max_model_len", "max_num_batched_tokens"}:
kwargs[name]["type"] = human_readable_int kwargs[name]["type"] = human_readable_int
elif contains_type(type_hints, float): elif contains_type(type_hints, float):
kwargs[name]["type"] = float kwargs[name]["type"] = float
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment