---
id: pr-flashinfer-3097
repo: flashinfer-ai/flashinfer
pr: 3097
title: Support NVFP4 KV for prefill and batch attention kernels
author: Tom-Zheng
date: '2026-04-28'
url: https://github.com/flashinfer-ai/flashinfer/pull/3097
source_category: upstream-code
architectures:
- sm100
tags:
- attention_kv
- cuda kernel
- fp4
- gemm_quant
- nvfp4
- paged attention
techniques:
- fine-grained-quantization
- persistent-kernels
- swizzling
hardware_features:
- fp4
- nvfp4
kernel_types:
- attention
- decode
- gemm
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 4e64219132b13aee43f9d516aba54160748a6feb
inclusion_reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.
changed_paths:
- csrc/xqa/utils.cuh
- flashinfer/attention/_core.py
- flashinfer/decode.py
- flashinfer/jit/attention/modules.py
- flashinfer/jit/utils.py
- flashinfer/prefill.py
- flashinfer/quantization/fp4_quantization.py
- flashinfer/utils.py
- include/flashinfer/attention/persistent.cuh
- include/flashinfer/attention/prefill.cuh
- include/flashinfer/cp_async.cuh
- include/flashinfer/frag_layout_swizzle.cuh
- include/flashinfer/permuted_smem.cuh
- include/flashinfer/vec_dtypes.cuh
- tests/attention/test_batch_attention.py
- tests/attention/test_batch_decode_kernels.py
- tests/attention/test_batch_prefill_kernels.py
- tests/attention/test_single_prefill.py
- tests/test_helpers/utils_fp4.py
artifact_dir: evidence/pull-bundles/flashinfer/gh-3097
head_ref_oid: 8e6bf6b34540e362c0165790bf1d33d100fa7f3f
---
# Support NVFP4 KV for prefill and batch attention kernels

- Repository: `flashinfer-ai/flashinfer`
- PR: [3097](https://github.com/flashinfer-ai/flashinfer/pull/3097)
- Decision: `Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.`
- Reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.

## Changed Paths

- `csrc/xqa/utils.cuh`
- `flashinfer/attention/_core.py`
- `flashinfer/decode.py`
- `flashinfer/jit/attention/modules.py`
- `flashinfer/jit/utils.py`
- `flashinfer/prefill.py`
- `flashinfer/quantization/fp4_quantization.py`
- `flashinfer/utils.py`
- `include/flashinfer/attention/persistent.cuh`
- `include/flashinfer/attention/prefill.cuh`
- `include/flashinfer/cp_async.cuh`
- `include/flashinfer/frag_layout_swizzle.cuh`
- `include/flashinfer/permuted_smem.cuh`
- `include/flashinfer/vec_dtypes.cuh`
- `tests/attention/test_batch_attention.py`
- `tests/attention/test_batch_decode_kernels.py`
- `tests/attention/test_batch_prefill_kernels.py`
- `tests/attention/test_single_prefill.py`
- `tests/test_helpers/utils_fp4.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-3097`
