"vllm/multimodal/processing/processor.py" did not exist on "0ad9951c416d33c5da4f7a504fb162cbe62386f5"
Unverified Commit 4fbd8bb5 authored by Harry Mellor's avatar Harry Mellor Committed by GitHub
Browse files

Fix passing `SpeculativeConfig` from the CLI (#22652)


Signed-off-by: default avatarHarry Mellor <19981378+hmellor@users.noreply.github.com>
parent ad344ef5
......@@ -826,6 +826,10 @@ class EngineArgs:
title="VllmConfig",
description=VllmConfig.__doc__,
)
# We construct SpeculativeConfig using fields from other configs in
# create_engine_config. So we set the type to a JSON string here to
# delay the Pydantic validation that comes with SpeculativeConfig.
vllm_kwargs["speculative_config"]["type"] = optional_type(json.loads)
vllm_group.add_argument("--speculative-config",
**vllm_kwargs["speculative_config"])
vllm_group.add_argument("--kv-transfer-config",
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment