Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
acb54ca8
Unverified
Commit
acb54ca8
authored
May 22, 2025
by
Rabi Mishra
Committed by
GitHub
May 21, 2025
Browse files
Intialize io_thread_pool attribute in the beginning. (#18331)
Signed-off-by:
rabi
<
ramishra@redhat.com
>
parent
6e0fd34d
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
vllm/v1/executor/multiproc_executor.py
vllm/v1/executor/multiproc_executor.py
+1
-1
No files found.
vllm/v1/executor/multiproc_executor.py
View file @
acb54ca8
...
...
@@ -50,6 +50,7 @@ class MultiprocExecutor(Executor):
self
.
is_failed
=
False
self
.
shutdown_event
=
threading
.
Event
()
self
.
failure_callback
:
Optional
[
FailureCallback
]
=
None
self
.
io_thread_pool
:
Optional
[
ThreadPoolExecutor
]
=
None
self
.
world_size
=
self
.
parallel_config
.
world_size
tensor_parallel_size
=
self
.
parallel_config
.
tensor_parallel_size
...
...
@@ -107,7 +108,6 @@ class MultiprocExecutor(Executor):
# For pipeline parallel, we use a thread pool for asynchronous
# execute_model.
self
.
io_thread_pool
:
Optional
[
ThreadPoolExecutor
]
=
None
if
self
.
max_concurrent_batches
>
1
:
# Note: must use only 1 IO thread to keep dequeue sequence
# from the response queue
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment