---
id: pr-flashinfer-3237
repo: flashinfer-ai/flashinfer
pr: 3237
title: 'perf: optimize per-token nvfp4 quantization kernel.'
author: IwakuraRein
date: '2026-05-08'
url: https://github.com/flashinfer-ai/flashinfer/pull/3237
source_category: upstream-code
architectures:
- sm100
tags:
- benchmark_test
- cuda kernel
- fp4
- gemm_quant
- nvfp4
techniques:
- fine-grained-quantization
hardware_features:
- fp4
- nvfp4
kernel_types:
- gemm
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 202af00dff85c1d5b50d15eab59d936365042af3
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- csrc/nv_internal/cpp/common/envUtils.cpp
- csrc/nv_internal/cpp/kernels/quantization.cu
- csrc/nv_internal/tensorrt_llm/common/envUtils.h
- csrc/nv_internal/tensorrt_llm/kernels/quantization.cuh
- csrc/nv_internal/tensorrt_llm/kernels/quantization.h
- csrc/nv_internal/tensorrt_llm/kernels/quantization_utils.cuh
- tests/utils/test_fp4_quantize.py
artifact_dir: evidence/pull-bundles/flashinfer/gh-3237
head_ref_oid: 62dadb3689bf99c9ae7f47e3efc0f2fad68995a0
---
# perf: optimize per-token nvfp4 quantization kernel.

- Repository: `flashinfer-ai/flashinfer`
- PR: [3237](https://github.com/flashinfer-ai/flashinfer/pull/3237)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `csrc/nv_internal/cpp/common/envUtils.cpp`
- `csrc/nv_internal/cpp/kernels/quantization.cu`
- `csrc/nv_internal/tensorrt_llm/common/envUtils.h`
- `csrc/nv_internal/tensorrt_llm/kernels/quantization.cuh`
- `csrc/nv_internal/tensorrt_llm/kernels/quantization.h`
- `csrc/nv_internal/tensorrt_llm/kernels/quantization_utils.cuh`
- `tests/utils/test_fp4_quantize.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-3237`
