"...git@developer.sourcefind.cn:2222/OpenDAS/vllm_cscc.git" did not exist on "c016c95b45765fcd432c533b16f6d17d77cc5f6d"
Unverified Commit eb1c9f24 authored by Hongkuan Zhou's avatar Hongkuan Zhou Committed by GitHub
Browse files

fix: always enable graceful shutdown as migration is not set in sglang (#2865)


Signed-off-by: default avatarhongkuanz <hongkuanz@nvidia.com>
parent 9524a464
......@@ -118,7 +118,7 @@ async def init(runtime: DistributedRuntime, config: Config):
await asyncio.gather(
generate_endpoint.serve_endpoint(
handler.generate,
graceful_shutdown == config.migration_limit <= 0,
graceful_shutdown=True,
metrics_labels=metrics_labels,
),
register_model(),
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment