Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
f218f9c2
Unverified
Commit
f218f9c2
authored
Jan 14, 2025
by
Rui Qiao
Committed by
GitHub
Jan 15, 2025
Browse files
[core] Turn off GPU communication overlap for Ray executor (#12051)
Signed-off-by:
Rui Qiao
<
ruisearch42@gmail.com
>
parent
0794e744
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
4 deletions
+4
-4
vllm/envs.py
vllm/envs.py
+4
-4
No files found.
vllm/envs.py
View file @
f218f9c2
...
...
@@ -45,7 +45,7 @@ if TYPE_CHECKING:
VLLM_USE_RAY_SPMD_WORKER
:
bool
=
False
VLLM_USE_RAY_COMPILED_DAG
:
bool
=
False
VLLM_USE_RAY_COMPILED_DAG_NCCL_CHANNEL
:
bool
=
True
VLLM_USE_RAY_COMPILED_DAG_OVERLAP_COMM
:
bool
=
Tru
e
VLLM_USE_RAY_COMPILED_DAG_OVERLAP_COMM
:
bool
=
Fals
e
VLLM_WORKER_MULTIPROC_METHOD
:
str
=
"fork"
VLLM_ASSETS_CACHE
:
str
=
os
.
path
.
join
(
VLLM_CACHE_ROOT
,
"assets"
)
VLLM_IMAGE_FETCH_TIMEOUT
:
int
=
5
...
...
@@ -340,11 +340,11 @@ environment_variables: Dict[str, Callable[[], Any]] = {
lambda
:
bool
(
int
(
os
.
getenv
(
"VLLM_USE_RAY_COMPILED_DAG_NCCL_CHANNEL"
,
"1"
))
),
# If the env var is set, it enables GPU communication overlap
in
# Ray's compiled DAG. This flag is ignored if
# If the env var is set, it enables GPU communication overlap
#
(experimental feature) in
Ray's compiled DAG. This flag is ignored if
# VLLM_USE_RAY_COMPILED_DAG is not set.
"VLLM_USE_RAY_COMPILED_DAG_OVERLAP_COMM"
:
lambda
:
bool
(
int
(
os
.
getenv
(
"VLLM_USE_RAY_COMPILED_DAG_OVERLAP_COMM"
,
"
1
"
))
lambda
:
bool
(
int
(
os
.
getenv
(
"VLLM_USE_RAY_COMPILED_DAG_OVERLAP_COMM"
,
"
0
"
))
),
# Use dedicated multiprocess context for workers.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment