Skip to content

GitLab

Unverified Commit 7b5a8b4a authored by Aaron Hao's avatar Aaron Hao Committed by GitHub
Browse files

[BUG] Reset running requests when clearing cache for pause/resume (#34382) 


Signed-off-by: default avatarhao-aaron <ahao@anyscale.com>
parent dea63512
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/v1/engine/async_llm.py
View file @ 7b5a8b4a
...@@ -793,7 +793,7 @@ class AsyncLLM(EngineClient): ...@@ -793,7 +793,7 @@ class AsyncLLM(EngineClient):
# Clear cache # Clear cache
if clear_cache: if clear_cache:
await self.reset_prefix_cache() await self.reset_prefix_cache(reset_running_requests=True)
await self.reset_mm_cache() await self.reset_mm_cache()
await self.reset_encoder_cache() await self.reset_encoder_cache()
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment