Unverified Commit 7717d083 authored by coolcloudcol's avatar coolcloudcol Committed by GitHub
Browse files

Fix an endless loop issue when engine_step throws a RuntimeError (#339)

parent 42e0c1df
...@@ -144,7 +144,11 @@ class AsyncLLMEngine: ...@@ -144,7 +144,11 @@ class AsyncLLMEngine:
# Kick the engine if the engine is not running. # Kick the engine if the engine is not running.
if not self.is_engine_running: if not self.is_engine_running:
await self.engine_step(request_id) try:
await self.engine_step(request_id)
except RuntimeError as e:
await self.abort(request_id)
raise e
# Wait for new output. The group_event will be set in engine_step # Wait for new output. The group_event will be set in engine_step
# when there is new output available for the sequence group. # when there is new output available for the sequence group.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment