Skip to content

GitLab

Unverified Commit 1cbcfb94 authored by Li, Jiang's avatar Li, Jiang Committed by GitHub
Browse files

[Bugfix][CPU] Skip unsupported custom op register on CPU (#25534) 


Signed-off-by: default avatarjiang1.li <jiang1.li@intel.com>
parent fed8a9b1
Show whitespace changes
Inline Side-by-side
Showing with 8 additions and 6 deletions
vllm/model_executor/layers/quantization/utils/fp8_utils.py
View file @ 1cbcfb94
......@@ -130,12 +130,14 @@ def _w8a8_triton_block_scaled_mm_fake(
device=qx.device)
direct_register_custom_op(
# Note: the check can be removed when CPU torch > 2.7
if not current_platform.is_cpu():
direct_register_custom_op(
"w8a8_triton_block_scaled_mm_func",
_w8a8_triton_block_scaled_mm_func,
fake_impl=_w8a8_triton_block_scaled_mm_fake,
dispatch_key="CUDA",
)
)
# TODO fix ROCm->Triton custom path:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment