Skip to content

GitLab

Unverified Commit bae888cb authored by Rui Qiao's avatar Rui Qiao Committed by GitHub
Browse files

[Bugfix] Clear engine reference in AsyncEngineRPCServer (#7618) 


Signed-off-by: default avatarRui Qiao <ruisearch42@gmail.com>
parent 6bd19551
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
vllm/entrypoints/openai/rpc/server.py
View file @ bae888cb
......@@ -38,6 +38,8 @@ class AsyncEngineRPCServer:
self.socket.close()
self.context.destroy()
self.engine.shutdown_background_loop()
# Clear the engine reference so that it can be GC'ed.
self.engine = None
async def get_model_config(self, identity):
"""Send the ModelConfig"""
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment