Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
73e3949d
Unverified
Commit
73e3949d
authored
Jul 24, 2025
by
Rui Qiao
Committed by
GitHub
Jul 24, 2025
Browse files
[Misc] Improve comment for DPEngineCoreActor._set_cuda_visible_devices() (#21501)
Signed-off-by:
Rui Qiao
<
ruisearch42@gmail.com
>
parent
6eca337c
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
7 additions
and
2 deletions
+7
-2
vllm/v1/engine/core.py
vllm/v1/engine/core.py
+7
-2
No files found.
vllm/v1/engine/core.py
View file @
73e3949d
...
@@ -1082,8 +1082,13 @@ class DPEngineCoreActor(DPEngineCoreProc):
...
@@ -1082,8 +1082,13 @@ class DPEngineCoreActor(DPEngineCoreProc):
# RAY_EXPERIMENTAL_NOSET_CUDA_VISIBLE_DEVICES, but vLLM workers created
# RAY_EXPERIMENTAL_NOSET_CUDA_VISIBLE_DEVICES, but vLLM workers created
# thereafter would have CUDA_VISIBLE_DEVICES set, which is sticky:
# thereafter would have CUDA_VISIBLE_DEVICES set, which is sticky:
# https://github.com/ray-project/ray/blob/e752fc319ddedd9779a0989b6d3613909bad75c9/python/ray/_private/worker.py#L456 # noqa: E501
# https://github.com/ray-project/ray/blob/e752fc319ddedd9779a0989b6d3613909bad75c9/python/ray/_private/worker.py#L456 # noqa: E501
# But vLLM worker assumes visibility into all local GPUs, therefore
# This is problematic because when the vLLM worker (a Ray actor)
# this results in incorrect indexing into the GPU ID list.
# executes a task, it indexes into the sticky CUDA_VISIBLE_DEVICES
# rather than directly using the GPU ID, potentially resulting in
# index out of bounds error. See:
# https://github.com/ray-project/ray/pull/40461/files#diff-31e8159767361e4bc259b6d9883d9c0d5e5db780fcea4a52ead4ee3ee4a59a78R1860 # noqa: E501
# and get_accelerator_ids_for_accelerator_resource() in worker.py
# of ray.
self
.
_set_cuda_visible_devices
(
vllm_config
,
local_dp_rank
)
self
.
_set_cuda_visible_devices
(
vllm_config
,
local_dp_rank
)
super
().
__init__
(
vllm_config
,
local_client
,
""
,
executor_class
,
super
().
__init__
(
vllm_config
,
local_client
,
""
,
executor_class
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment