---
id: pr-tensorrt-llm-4867
repo: NVIDIA/TensorRT-LLM
pr: 4867
title: 'feat: Add w4a8_mxfp4_fp8 quantization recipe.'
author: Tracin
date: '2025-06-16'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/4867
source_category: upstream-code
architectures:
- sm100
tags:
- gemm_quant
- mxfp4
techniques:
- cache-policy
- fine-grained-quantization
hardware_features:
- blackwell
- fp4
- fp8
- mxfp4
- nvfp4
kernel_types:
- gemm
languages:
- cuda-cpp
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: ef3fdc80516f86b6c610c377f68e0b3f2d6528b9
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/fp4_gemm_bf16.cu
- cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/fp4_gemm_fp16.cu
- cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/fp4_gemm_fp32.cu
- cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/fp4_gemm_template.h
- cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/mxfp8_mxfp4_gemm_template_sm100.h
- cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/nvfp4_nvfp4_gemm_template_sm100.h
- cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/nvfp4_nvfp4_gemm_template_sm120.h
- cpp/tensorrt_llm/kernels/cutlass_kernels/include/fp4_gemm.h
- cpp/tensorrt_llm/kernels/internal_cutlass_kernels/aarch64-linux-gnu/tensorrt_llm_internal_cutlass_kernels_static.tar.xz
- cpp/tensorrt_llm/kernels/internal_cutlass_kernels/aarch64-linux-gnu/version.txt
- cpp/tensorrt_llm/kernels/internal_cutlass_kernels/include/fp4_gemm.h
- cpp/tensorrt_llm/kernels/internal_cutlass_kernels/x86_64-linux-gnu/tensorrt_llm_internal_cutlass_kernels_static.tar.xz
- cpp/tensorrt_llm/kernels/internal_cutlass_kernels/x86_64-linux-gnu/version.txt
- cpp/tensorrt_llm/thop/fp4Gemm.cpp
- tensorrt_llm/_torch/auto_deploy/custom_ops/quant.py
- tensorrt_llm/_torch/compilation/patterns/ub_allreduce.py
- tests/unittest/_torch/thop/test_fp4_bmm_quantize.py
- tests/unittest/_torch/thop/test_fp4_gemm_quantize.py
- tests/unittest/_torch/thop/test_fp4_linear.py
- tests/unittest/_torch/thop/test_w4a8_mxfp4_mxfp8_gemm.py
- tensorrt_llm/_torch/custom_ops/torch_custom_ops.py
- tensorrt_llm/_torch/modules/linear.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-4867
head_ref_oid: bba771d52a55eba8bbae94710587a843681f2e33
---
# feat: Add w4a8_mxfp4_fp8 quantization recipe.

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [4867](https://github.com/NVIDIA/TensorRT-LLM/pull/4867)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/fp4_gemm_bf16.cu`
- `cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/fp4_gemm_fp16.cu`
- `cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/fp4_gemm_fp32.cu`
- `cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/fp4_gemm_template.h`
- `cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/mxfp8_mxfp4_gemm_template_sm100.h`
- `cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/nvfp4_nvfp4_gemm_template_sm100.h`
- `cpp/tensorrt_llm/kernels/cutlass_kernels/fp4_gemm/nvfp4_nvfp4_gemm_template_sm120.h`
- `cpp/tensorrt_llm/kernels/cutlass_kernels/include/fp4_gemm.h`
- `cpp/tensorrt_llm/kernels/internal_cutlass_kernels/aarch64-linux-gnu/tensorrt_llm_internal_cutlass_kernels_static.tar.xz`
- `cpp/tensorrt_llm/kernels/internal_cutlass_kernels/aarch64-linux-gnu/version.txt`
- `cpp/tensorrt_llm/kernels/internal_cutlass_kernels/include/fp4_gemm.h`
- `cpp/tensorrt_llm/kernels/internal_cutlass_kernels/x86_64-linux-gnu/tensorrt_llm_internal_cutlass_kernels_static.tar.xz`
- `cpp/tensorrt_llm/kernels/internal_cutlass_kernels/x86_64-linux-gnu/version.txt`
- `cpp/tensorrt_llm/thop/fp4Gemm.cpp`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/quant.py`
- `tensorrt_llm/_torch/compilation/patterns/ub_allreduce.py`
- `tests/unittest/_torch/thop/test_fp4_bmm_quantize.py`
- `tests/unittest/_torch/thop/test_fp4_gemm_quantize.py`
- `tests/unittest/_torch/thop/test_fp4_linear.py`
- `tests/unittest/_torch/thop/test_w4a8_mxfp4_mxfp8_gemm.py`
- `tensorrt_llm/_torch/custom_ops/torch_custom_ops.py`
- `tensorrt_llm/_torch/modules/linear.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-4867`
