Unverified Commit 89a55418 authored by shangmingc's avatar shangmingc Committed by GitHub
Browse files

[PD] Fix unclosed prefill connection warning of mini_lb (#5155)


Signed-off-by: default avatarShangming Cai <caishangming@linux.alibaba.com>
parent 2695ab05
......@@ -232,6 +232,9 @@ async def handle_generate_request(request_data: dict):
yield b"data: " + orjson.dumps(
error_msg, option=orjson.OPT_NON_STR_KEYS
) + b"\n\n"
finally:
if prefill_response is not None:
await prefill_response.release()
return StreamingResponse(
stream_results(),
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment