Skip to content

GitLab

Unverified Commit 499b074b authored by Ning Xie's avatar Ning Xie Committed by GitHub
Browse files

[Misc] refactor code by import as for torch._inductor.config (#23677) 


Signed-off-by: default avatarAndy Xie <andy.xning@gmail.com>
parent ff0e59d8
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 4 deletions
vllm/v1/worker/cpu_model_runner.py
View file @ 499b074b
...@@ -138,14 +138,14 @@ def _torch_cuda_wrapper(): ...@@ -138,14 +138,14 @@ def _torch_cuda_wrapper():
@contextmanager @contextmanager
def _set_global_compilation_settings(config: VllmConfig): def _set_global_compilation_settings(config: VllmConfig):
import torch._inductor.config import torch._inductor.config as torch_inductor_config
inductor_config = config.compilation_config.inductor_compile_config inductor_config = config.compilation_config.inductor_compile_config
# Note: The MKLDNN and CPPGEMM backend requires freezing parameters. # Note: The MKLDNN and CPPGEMM backend requires freezing parameters.
freezing_value = torch._inductor.config.freezing freezing_value = torch_inductor_config.freezing
try: try:
if inductor_config.get("max_autotune", False): if inductor_config.get("max_autotune", False):
torch._inductor.config.freezing = True torch_inductor_config.freezing = True
yield yield
finally: finally:
torch._inductor.config.freezing = freezing_value torch_inductor_config.freezing = freezing_value
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment