Skip to content

GitLab

Unverified Commit f29a718f authored by shangmingc's avatar shangmingc Committed by GitHub
Browse files

[PD] Fix generate endpoint of min_lb for PD (#5598) 


Signed-off-by: default avatarShangming Cai <caishangming@linux.alibaba.com>
parent 3f57b00a
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
python/sglang/srt/disaggregation/mini_lb.py
View file @ f29a718f
......@@ -187,11 +187,11 @@ async def handle_generate_request(request_data: dict):
if request_data.get("stream", False):
return await load_balancer.generate_stream(
modified_request, prefill_server, decode_server
modified_request, prefill_server, decode_server, "generate"
)
else:
return await load_balancer.generate(
modified_request, prefill_server, decode_server
modified_request, prefill_server, decode_server, "generate"
)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment