---
id: pr-vllm-11868
repo: vllm-project/vllm
pr: 11868
title: '[Kernel] Update `cutlass_scaled_mm` to support 2d group (blockwise) scaling'
author: LucasWilkinson
date: '2025-01-31'
url: https://github.com/vllm-project/vllm/pull/11868
source_category: upstream-code
architectures:
- sm100
tags:
- fp8
- gemm
- quantization
- tma
techniques: []
hardware_features:
- fp8
- tma
kernel_types:
- gemm
- quantization
languages:
- cuda-cpp
captured_at: '2026-04-27'
status: merged
merge_sha: 9798b2fb0052092a6420172e41c0c8a307eedfa6
inclusion_reason: kernel file changes
changed_paths:
- CMakeLists.txt
- benchmarks/cutlass_benchmarks/w8a8_benchmarks.py
- csrc/core/math.hpp
- csrc/cutlass_extensions/common.hpp
- csrc/cutlass_extensions/gemm/collective/collective_builder.hpp
- csrc/cutlass_extensions/gemm/collective/fp8_accumulation.hpp
- csrc/cutlass_extensions/gemm/collective/sm90_mma_tma_gmma_ss_warpspecialized_fp8_blockwise_scaling.hpp
- csrc/cutlass_extensions/gemm/dispatch_policy.hpp
- csrc/cutlass_extensions/vllm_collective_builder.cuh
- csrc/quantization/cutlass_w8a8/c3x/cutlass_gemm_caller.cuh
- csrc/quantization/cutlass_w8a8/c3x/scaled_mm.cuh
- csrc/quantization/cutlass_w8a8/c3x/scaled_mm_azp_sm90_int8.cu
- csrc/quantization/cutlass_w8a8/c3x/scaled_mm_blockwise_sm90_fp8.cu
- csrc/quantization/cutlass_w8a8/c3x/scaled_mm_blockwise_sm90_fp8_dispatch.cuh
- csrc/quantization/cutlass_w8a8/c3x/scaled_mm_kernels.hpp
- csrc/quantization/cutlass_w8a8/c3x/scaled_mm_sm90_fp8.cu
- csrc/quantization/cutlass_w8a8/c3x/scaled_mm_sm90_fp8_dispatch.cuh
- csrc/quantization/cutlass_w8a8/c3x/scaled_mm_sm90_int8.cu
- csrc/quantization/cutlass_w8a8/c3x/scaled_mm_sm90_int8_dispatch.cuh
- csrc/quantization/cutlass_w8a8/scaled_mm_c3x.cu
- csrc/quantization/cutlass_w8a8/scaled_mm_entry.cu
- csrc/quantization/machete/machete_mainloop.cuh
- tests/kernels/test_cutlass.py
- tests/kernels/utils.py
- vllm/_custom_ops.py
head_ref_oid: 69778419c92db9989014af225ac4ca492e26de36
artifact_dir: evidence/pull-bundles/vllm/gh-11868
---
# [Kernel] Update `cutlass_scaled_mm` to support 2d group (blockwise) scaling

- Repository: `vllm-project/vllm`
- PR: [11868](https://github.com/vllm-project/vllm/pull/11868)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `CMakeLists.txt`
- `benchmarks/cutlass_benchmarks/w8a8_benchmarks.py`
- `csrc/core/math.hpp`
- `csrc/cutlass_extensions/common.hpp`
- `csrc/cutlass_extensions/gemm/collective/collective_builder.hpp`
- `csrc/cutlass_extensions/gemm/collective/fp8_accumulation.hpp`
- `csrc/cutlass_extensions/gemm/collective/sm90_mma_tma_gmma_ss_warpspecialized_fp8_blockwise_scaling.hpp`
- `csrc/cutlass_extensions/gemm/dispatch_policy.hpp`
- `csrc/cutlass_extensions/vllm_collective_builder.cuh`
- `csrc/quantization/cutlass_w8a8/c3x/cutlass_gemm_caller.cuh`
- `csrc/quantization/cutlass_w8a8/c3x/scaled_mm.cuh`
- `csrc/quantization/cutlass_w8a8/c3x/scaled_mm_azp_sm90_int8.cu`
- `csrc/quantization/cutlass_w8a8/c3x/scaled_mm_blockwise_sm90_fp8.cu`
- `csrc/quantization/cutlass_w8a8/c3x/scaled_mm_blockwise_sm90_fp8_dispatch.cuh`
- `csrc/quantization/cutlass_w8a8/c3x/scaled_mm_kernels.hpp`
- `csrc/quantization/cutlass_w8a8/c3x/scaled_mm_sm90_fp8.cu`
- `csrc/quantization/cutlass_w8a8/c3x/scaled_mm_sm90_fp8_dispatch.cuh`
- `csrc/quantization/cutlass_w8a8/c3x/scaled_mm_sm90_int8.cu`
- `csrc/quantization/cutlass_w8a8/c3x/scaled_mm_sm90_int8_dispatch.cuh`
- `csrc/quantization/cutlass_w8a8/scaled_mm_c3x.cu`
- `csrc/quantization/cutlass_w8a8/scaled_mm_entry.cu`
- `csrc/quantization/machete/machete_mainloop.cuh`
- `tests/kernels/test_cutlass.py`
- `tests/kernels/utils.py`
- `vllm/_custom_ops.py`

## Evidence Bundle

`evidence/pull-bundles/vllm/gh-11868`
