---
id: pr-flashinfer-1033
repo: flashinfer-ai/flashinfer
pr: 1033
title: 'feat: add functional per-head FP8 quantization for FA3'
author: happierpig
date: '2025-04-29'
url: https://github.com/flashinfer-ai/flashinfer/pull/1033
source_category: upstream-code
architectures:
- sm100
- sm90
tags:
- attention
- decode
- epilogue-fusion
- fp8
- prefill
- quantization
techniques:
- epilogue-fusion
hardware_features:
- fp8
kernel_types:
- attention
- decode
- prefill
- quantization
languages:
- cuda-cpp
captured_at: '2026-04-17'
status: merged
merge_sha: 116d97d7fcd2df0691cacf049e5923340d3af9d8
inclusion_reason: kernel file changes
changed_paths:
- benchmarks/bench_hopper_fp8_attention.py
- csrc/batch_prefill_fp8_paged_sm90_kernel_inst.jinja
- csrc/batch_prefill_fp8_ragged_sm90_kernel_inst.jinja
- csrc/batch_prefill_fp8_sm90.cu
- csrc/single_prefill_fp8_sm90.cu
- csrc/single_prefill_fp8_sm90_kernel_inst.jinja
- flashinfer/decode.py
- flashinfer/jit/attention/pytorch.py
- flashinfer/prefill.py
- flashinfer/sparse.py
- include/flashinfer/attention/hopper/quantization/epilogue.cuh
- include/flashinfer/attention/hopper/quantization/kernel_traits.cuh
- include/flashinfer/attention/hopper/quantization/mainloop_load.cuh
- include/flashinfer/attention/hopper/quantization/mainloop_sparse_load.cuh
- include/flashinfer/attention/hopper/quantization/prefill_sm90.cuh
- src/fp8-dev/CMakeLists.txt
- src/fp8-dev/bench_single_prefill_sm90.cu
- src/fp8-dev/cpu_reference.h
- src/fp8-dev/flashattention_ops.h
- src/fp8-dev/flashinfer_ops.cu
- src/fp8-dev/test_single_prefill_fa3_sm90.cu
- src/fp8-dev/utils.h
- tests/test_hopper_fp8_attention.py
head_ref_oid: 589fa3e822e19475c3de3a1133d25a8584ca76c8
artifact_dir: evidence/pull-bundles/flashinfer/gh-1033
---
# feat: add functional per-head FP8 quantization for FA3

- Repository: `flashinfer-ai/flashinfer`
- PR: [1033](https://github.com/flashinfer-ai/flashinfer/pull/1033)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `benchmarks/bench_hopper_fp8_attention.py`
- `csrc/batch_prefill_fp8_paged_sm90_kernel_inst.jinja`
- `csrc/batch_prefill_fp8_ragged_sm90_kernel_inst.jinja`
- `csrc/batch_prefill_fp8_sm90.cu`
- `csrc/single_prefill_fp8_sm90.cu`
- `csrc/single_prefill_fp8_sm90_kernel_inst.jinja`
- `flashinfer/decode.py`
- `flashinfer/jit/attention/pytorch.py`
- `flashinfer/prefill.py`
- `flashinfer/sparse.py`
- `include/flashinfer/attention/hopper/quantization/epilogue.cuh`
- `include/flashinfer/attention/hopper/quantization/kernel_traits.cuh`
- `include/flashinfer/attention/hopper/quantization/mainloop_load.cuh`
- `include/flashinfer/attention/hopper/quantization/mainloop_sparse_load.cuh`
- `include/flashinfer/attention/hopper/quantization/prefill_sm90.cuh`
- `src/fp8-dev/CMakeLists.txt`
- `src/fp8-dev/bench_single_prefill_sm90.cu`
- `src/fp8-dev/cpu_reference.h`
- `src/fp8-dev/flashattention_ops.h`
- `src/fp8-dev/flashinfer_ops.cu`
- `src/fp8-dev/test_single_prefill_fa3_sm90.cu`
- `src/fp8-dev/utils.h`
- `tests/test_hopper_fp8_attention.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-1033`
