---
id: pr-vllm-7701
repo: vllm-project/vllm
pr: 7701
title: '[Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin'
author: LucasWilkinson
date: '2024-09-23'
url: https://github.com/vllm-project/vllm/pull/7701
source_category: upstream-code
architectures:
- sm100
tags:
- Machete
- benchmark_test
- gemm_quant
techniques:
- fine-grained-quantization
hardware_features: []
kernel_types:
- gemm
- moe
languages:
- cuda-cpp
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: 86e9c8df29a954a7a2fc46e9985fecc2a2e15ae8
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- CMakeLists.txt
- benchmarks/kernels/benchmark_machete.py
- benchmarks/kernels/requirements.txt
- csrc/cutlass_extensions/torch_utils.hpp
- csrc/moe/marlin_moe_ops.cu
- csrc/ops.h
- csrc/permute_cols.cu
- csrc/quantization/machete/generate.py
- csrc/quantization/machete/machete_mm_kernel.cuh
- csrc/quantization/machete/machete_mm_launcher.cuh
- csrc/quantization/machete/machete_prepack_launcher.cuh
- csrc/torch_bindings.cpp
- tests/kernels/test_machete_gemm.py
- tests/kernels/test_permute_cols.py
- vllm/_custom_ops.py
- vllm/model_executor/layers/quantization/awq_marlin.py
- vllm/model_executor/layers/quantization/compressed_tensors/schemes/compressed_tensors_wNa16.py
- vllm/model_executor/layers/quantization/gptq_marlin.py
- vllm/model_executor/layers/quantization/kernels/MPLinearKernel.py
- vllm/model_executor/layers/quantization/kernels/__init__.py
- vllm/model_executor/layers/quantization/kernels/machete.py
- vllm/model_executor/layers/quantization/kernels/marlin.py
- vllm/model_executor/layers/quantization/utils/__init__.py
- vllm/model_executor/layers/quantization/utils/layer_utils.py
- vllm/model_executor/layers/quantization/utils/machete_utils.py
- vllm/model_executor/layers/quantization/utils/marlin_utils.py
- vllm/model_executor/layers/quantization/utils/quant_utils.py
- vllm/model_executor/models/dbrx.py
- vllm/model_executor/parameter.py
artifact_dir: evidence/pull-bundles/vllm/gh-7701
head_ref_oid: 05752e90813448b6fe15941e8946ef3157df6799
---
# [Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin

- Repository: `vllm-project/vllm`
- PR: [7701](https://github.com/vllm-project/vllm/pull/7701)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `CMakeLists.txt`
- `benchmarks/kernels/benchmark_machete.py`
- `benchmarks/kernels/requirements.txt`
- `csrc/cutlass_extensions/torch_utils.hpp`
- `csrc/moe/marlin_moe_ops.cu`
- `csrc/ops.h`
- `csrc/permute_cols.cu`
- `csrc/quantization/machete/generate.py`
- `csrc/quantization/machete/machete_mm_kernel.cuh`
- `csrc/quantization/machete/machete_mm_launcher.cuh`
- `csrc/quantization/machete/machete_prepack_launcher.cuh`
- `csrc/torch_bindings.cpp`
- `tests/kernels/test_machete_gemm.py`
- `tests/kernels/test_permute_cols.py`
- `vllm/_custom_ops.py`
- `vllm/model_executor/layers/quantization/awq_marlin.py`
- `vllm/model_executor/layers/quantization/compressed_tensors/schemes/compressed_tensors_wNa16.py`
- `vllm/model_executor/layers/quantization/gptq_marlin.py`
- `vllm/model_executor/layers/quantization/kernels/MPLinearKernel.py`
- `vllm/model_executor/layers/quantization/kernels/__init__.py`
- `vllm/model_executor/layers/quantization/kernels/machete.py`
- `vllm/model_executor/layers/quantization/kernels/marlin.py`
- `vllm/model_executor/layers/quantization/utils/__init__.py`
- `vllm/model_executor/layers/quantization/utils/layer_utils.py`
- `vllm/model_executor/layers/quantization/utils/machete_utils.py`
- `vllm/model_executor/layers/quantization/utils/marlin_utils.py`
- `vllm/model_executor/layers/quantization/utils/quant_utils.py`
- `vllm/model_executor/models/dbrx.py`
- `vllm/model_executor/parameter.py`

## Evidence Bundle

`evidence/pull-bundles/vllm/gh-7701`
