Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
a9fe0793
Unverified
Commit
a9fe0793
authored
Oct 29, 2025
by
Boyuan Feng
Committed by
GitHub
Oct 29, 2025
Browse files
`use_aot_compile` should respect `VLLM_DISABLE_COMPILE_CACHE` (#27698)
Signed-off-by:
Boyuan Feng
<
boyuan@meta.com
>
parent
7568a282
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
11 additions
and
4 deletions
+11
-4
vllm/envs.py
vllm/envs.py
+11
-4
No files found.
vllm/envs.py
View file @
a9fe0793
...
@@ -247,10 +247,19 @@ def maybe_convert_bool(value: str | None) -> bool | None:
...
@@ -247,10 +247,19 @@ def maybe_convert_bool(value: str | None) -> bool | None:
return
bool
(
int
(
value
))
return
bool
(
int
(
value
))
def
disable_compile_cache
()
->
bool
:
return
bool
(
int
(
os
.
getenv
(
"VLLM_DISABLE_COMPILE_CACHE"
,
"0"
)))
def
use_aot_compile
()
->
bool
:
def
use_aot_compile
()
->
bool
:
from
vllm.utils.torch_utils
import
is_torch_equal_or_newer
from
vllm.utils.torch_utils
import
is_torch_equal_or_newer
default_value
=
"1"
if
is_torch_equal_or_newer
(
"2.10.0.dev"
)
else
"0"
default_value
=
(
"1"
if
is_torch_equal_or_newer
(
"2.10.0.dev"
)
and
not
disable_compile_cache
()
else
"0"
)
return
os
.
environ
.
get
(
"VLLM_USE_AOT_COMPILE"
,
default_value
)
==
"1"
return
os
.
environ
.
get
(
"VLLM_USE_AOT_COMPILE"
,
default_value
)
==
"1"
...
@@ -963,9 +972,7 @@ environment_variables: dict[str, Callable[[], Any]] = {
...
@@ -963,9 +972,7 @@ environment_variables: dict[str, Callable[[], Any]] = {
"VLLM_LOG_BATCHSIZE_INTERVAL"
:
lambda
:
float
(
"VLLM_LOG_BATCHSIZE_INTERVAL"
:
lambda
:
float
(
os
.
getenv
(
"VLLM_LOG_BATCHSIZE_INTERVAL"
,
"-1"
)
os
.
getenv
(
"VLLM_LOG_BATCHSIZE_INTERVAL"
,
"-1"
)
),
),
"VLLM_DISABLE_COMPILE_CACHE"
:
lambda
:
bool
(
"VLLM_DISABLE_COMPILE_CACHE"
:
disable_compile_cache
,
int
(
os
.
getenv
(
"VLLM_DISABLE_COMPILE_CACHE"
,
"0"
))
),
# If set, vllm will run in development mode, which will enable
# If set, vllm will run in development mode, which will enable
# some additional endpoints for developing and debugging,
# some additional endpoints for developing and debugging,
# e.g. `/reset_prefix_cache`
# e.g. `/reset_prefix_cache`
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment