---
id: pr-vllm-10995
repo: vllm-project/vllm
pr: 10995
title: '[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support'
author: dsikka
date: '2024-12-18'
url: https://github.com/vllm-project/vllm/pull/10995
source_category: upstream-code
architectures:
- sm100
tags:
- Machete
- benchmark_test
- gemm_quant
techniques:
- epilogue-fusion
- fine-grained-quantization
hardware_features:
- fp8
kernel_types:
- gemm
languages:
- cuda-cpp
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: 60508ffda91c22e4cde3b18f149d222211db8886
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- CMakeLists.txt
- benchmarks/cutlass_benchmarks/sparse_benchmarks.py
- benchmarks/cutlass_benchmarks/utils.py
- benchmarks/cutlass_benchmarks/w8a8_benchmarks.py
- benchmarks/cutlass_benchmarks/weight_shapes.py
- csrc/core/math.hpp
- csrc/cutlass_extensions/common.cpp
- csrc/cutlass_extensions/common.hpp
- csrc/cutlass_extensions/epilogue/scaled_mm_epilogues_c3x.hpp
- csrc/ops.h
- csrc/quantization/cutlass_w8a8/common.hpp
- csrc/quantization/cutlass_w8a8/scaled_mm_c2x.cuh
- csrc/quantization/cutlass_w8a8/scaled_mm_c3x.cu
- csrc/quantization/cutlass_w8a8/scaled_mm_entry.cu
- csrc/sparse/cutlass/sparse_compressor_c3x.cu
- csrc/sparse/cutlass/sparse_compressor_entry.cu
- csrc/sparse/cutlass/sparse_scaled_mm_c3x.cu
- csrc/sparse/cutlass/sparse_scaled_mm_c3x.cuh
- csrc/sparse/cutlass/sparse_scaled_mm_entry.cu
- csrc/torch_bindings.cpp
- pyproject.toml
- tests/kernels/test_semi_structured.py
- tests/quantization/test_compressed_tensors.py
- tests/weight_loading/models.txt
- tests/weight_loading/run_model_weight_loading_test.sh
- tests/weight_loading/test_weight_loading.py
- vllm/_custom_ops.py
- vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors.py
- vllm/model_executor/layers/quantization/compressed_tensors/schemes/__init__.py
- vllm/model_executor/layers/quantization/compressed_tensors/schemes/compressed_tensors_24.py
artifact_dir: evidence/pull-bundles/vllm/gh-10995
head_ref_oid: 2c15abcb6f1ffddfd2989a5a05c165022ef5fe78
---
# [Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support

- Repository: `vllm-project/vllm`
- PR: [10995](https://github.com/vllm-project/vllm/pull/10995)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `CMakeLists.txt`
- `benchmarks/cutlass_benchmarks/sparse_benchmarks.py`
- `benchmarks/cutlass_benchmarks/utils.py`
- `benchmarks/cutlass_benchmarks/w8a8_benchmarks.py`
- `benchmarks/cutlass_benchmarks/weight_shapes.py`
- `csrc/core/math.hpp`
- `csrc/cutlass_extensions/common.cpp`
- `csrc/cutlass_extensions/common.hpp`
- `csrc/cutlass_extensions/epilogue/scaled_mm_epilogues_c3x.hpp`
- `csrc/ops.h`
- `csrc/quantization/cutlass_w8a8/common.hpp`
- `csrc/quantization/cutlass_w8a8/scaled_mm_c2x.cuh`
- `csrc/quantization/cutlass_w8a8/scaled_mm_c3x.cu`
- `csrc/quantization/cutlass_w8a8/scaled_mm_entry.cu`
- `csrc/sparse/cutlass/sparse_compressor_c3x.cu`
- `csrc/sparse/cutlass/sparse_compressor_entry.cu`
- `csrc/sparse/cutlass/sparse_scaled_mm_c3x.cu`
- `csrc/sparse/cutlass/sparse_scaled_mm_c3x.cuh`
- `csrc/sparse/cutlass/sparse_scaled_mm_entry.cu`
- `csrc/torch_bindings.cpp`
- `pyproject.toml`
- `tests/kernels/test_semi_structured.py`
- `tests/quantization/test_compressed_tensors.py`
- `tests/weight_loading/models.txt`
- `tests/weight_loading/run_model_weight_loading_test.sh`
- `tests/weight_loading/test_weight_loading.py`
- `vllm/_custom_ops.py`
- `vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors.py`
- `vllm/model_executor/layers/quantization/compressed_tensors/schemes/__init__.py`
- `vllm/model_executor/layers/quantization/compressed_tensors/schemes/compressed_tensors_24.py`

## Evidence Bundle

`evidence/pull-bundles/vllm/gh-10995`
