Skip to content

GitLab

Unverified Commit 554df8a6 authored by Xiao's avatar Xiao Committed by GitHub
Browse files

Revert "[compile][startup] Disable C++ compilation of symbolic shapes" (#22122) 


Signed-off-by: default avatarXiao Liu <xiszishu@gmail.com>
parent 73e1b9b1
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 9 deletions
vllm/compilation/decorators.py
View file @ 554df8a6
...@@ -267,15 +267,8 @@ def _support_torch_compile( ...@@ -267,15 +267,8 @@ def _support_torch_compile(
code.co_filename) code.co_filename)
return inline_call(parent, func, args, kwargs) return inline_call(parent, func, args, kwargs)
# Disable the C++ compilation of symbolic shape guards. C++-fication with patch.object(InliningInstructionTranslator, 'inline_call',
# of symbolic shape guards can improve guard overhead. But, since patched_inline_call):
# vllm skip guards anyways, setting this flag to False can improve
# compile time.
with torch._dynamo.config.patch("enable_cpp_symbolic_shape_guards",
False), patch.object(
InliningInstructionTranslator,
'inline_call',
patched_inline_call):
output = self.compiled_callable(*args, **kwargs) output = self.compiled_callable(*args, **kwargs)
return output return output
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment