Skip to content

GitLab

Unverified Commit 8c25f9cf authored by Boyuan Feng's avatar Boyuan Feng Committed by GitHub
Browse files

[BugFix] skip combo kernel on cpu (#29129) 


Signed-off-by: default avatarBoyuan Feng <boyuan@meta.com>
parent 56e96b37
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
vllm/config/compilation.py
View file @ 8c25f9cf
...@@ -664,6 +664,8 @@ class CompilationConfig: ...@@ -664,6 +664,8 @@ class CompilationConfig:
is_torch_equal_or_newer("2.9.0.dev") is_torch_equal_or_newer("2.9.0.dev")
and "combo_kernels" not in self.inductor_compile_config and "combo_kernels" not in self.inductor_compile_config
and "benchmark_combo_kernel" not in self.inductor_compile_config and "benchmark_combo_kernel" not in self.inductor_compile_config
# (fixme @boyuan) combo kernel does not support cpu yet.
and not current_platform.is_cpu()
): ):
# use horizontal fusion, which is useful for fusing qk-norm and # use horizontal fusion, which is useful for fusing qk-norm and
# qk-rope when query and key have different shapes. # qk-rope when query and key have different shapes.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment