Unverified Commit 9659bc7f authored by Animesh Jain's avatar Animesh Jain Committed by GitHub
Browse files

[compile][startup] Disable C++ compilation of symbolic shapes (#20836)


Signed-off-by: default avatarAnimesh Jain <anijain@umich.edu>
parent 3277e8f9
...@@ -267,7 +267,14 @@ def _support_torch_compile( ...@@ -267,7 +267,14 @@ def _support_torch_compile(
code.co_filename) code.co_filename)
return inline_call(parent, func, args, kwargs) return inline_call(parent, func, args, kwargs)
with patch.object(InliningInstructionTranslator, 'inline_call', # Disable the C++ compilation of symbolic shape guards. C++-fication
# of symbolic shape guards can improve guard overhead. But, since
# vllm skip guards anyways, setting this flag to False can improve
# compile time.
with torch._dynamo.config.patch("enable_cpp_symbolic_shape_guards",
False), patch.object(
InliningInstructionTranslator,
'inline_call',
patched_inline_call): patched_inline_call):
output = self.compiled_callable(*args, **kwargs) output = self.compiled_callable(*args, **kwargs)
return output return output
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment