Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
norm
vllm
Commits
7717d083
"docs/source/ja/index.md" did not exist on "48d0123f0f4415b1bb78f5a538df8b0b9975c6d4"
Unverified
Commit
7717d083
authored
Jul 04, 2023
by
coolcloudcol
Committed by
GitHub
Jul 03, 2023
Browse files
Fix an endless loop issue when engine_step throws a RuntimeError (#339)
parent
42e0c1df
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
1 deletion
+5
-1
vllm/engine/async_llm_engine.py
vllm/engine/async_llm_engine.py
+5
-1
No files found.
vllm/engine/async_llm_engine.py
View file @
7717d083
...
...
@@ -144,7 +144,11 @@ class AsyncLLMEngine:
# Kick the engine if the engine is not running.
if
not
self
.
is_engine_running
:
await
self
.
engine_step
(
request_id
)
try
:
await
self
.
engine_step
(
request_id
)
except
RuntimeError
as
e
:
await
self
.
abort
(
request_id
)
raise
e
# Wait for new output. The group_event will be set in engine_step
# when there is new output available for the sequence group.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment