Skip to content

GitLab

Unverified Commit 7c0cf3bc authored by Sage Moore's avatar Sage Moore Committed by GitHub
Browse files

Cap the number of API servers to 1 when using Elastic EP. (#37466) 


Signed-off-by: default avatarSage Moore <sage@neuralmagic.com>
parent 572b4329
Show whitespace changes
Inline Side-by-side
Showing with 9 additions and 0 deletions
vllm/entrypoints/cli/serve.py
View file @ 7c0cf3bc
...@@ -108,6 +108,15 @@ class ServeSubcommand(CLISubcommand): ...@@ -108,6 +108,15 @@ class ServeSubcommand(CLISubcommand):
args.api_server_count, args.api_server_count,
) )
# Elastic EP currently only supports running with at most one API server.
if getattr(args, "enable_elastic_ep", False) and args.api_server_count > 1:
logger.warning(
"Elastic EP only supports running with with at most one API server. "
"Capping api_server_count from %d to 1.",
args.api_server_count,
)
args.api_server_count = 1
if args.api_server_count < 1: if args.api_server_count < 1:
run_headless(args) run_headless(args)
elif args.api_server_count > 1: elif args.api_server_count > 1:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment