---
id: pr-flashinfer-3129
repo: flashinfer-ai/flashinfer
pr: 3129
title: 'feat: Enable FP8 (E4M3/E5M2) in concat_mla_k for optimize long-context prefill performance and refactor type dispatch for BF16/FP16'
author: qiching
date: '2026-05-07'
url: https://github.com/flashinfer-ai/flashinfer/pull/3129
source_category: upstream-code
architectures:
- sm100
tags:
- attention_kv
- benchmark_test
- fp8
- gemm_quant
- mla
techniques:
- fine-grained-quantization
hardware_features:
- fp8
kernel_types:
- attention
- gemm
- mla
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 8d1b69b2b4b73a04895dacc7486bec25416e93c4
inclusion_reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.
changed_paths:
- csrc/concat_mla.cu
- csrc/tvm_ffi_utils.h
- flashinfer/concat_ops.py
- include/flashinfer/concat_mla.cuh
- include/flashinfer/utils.cuh
- tests/utils/test_concat_mla.py
artifact_dir: evidence/pull-bundles/flashinfer/gh-3129
head_ref_oid: 896d3ac733b3b56e0931be324f1e26889a608cb0
---
# feat: Enable FP8 (E4M3/E5M2) in concat_mla_k for optimize long-context prefill performance and refactor type dispatch for BF16/FP16

- Repository: `flashinfer-ai/flashinfer`
- PR: [3129](https://github.com/flashinfer-ai/flashinfer/pull/3129)
- Decision: `Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.`
- Reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.

## Changed Paths

- `csrc/concat_mla.cu`
- `csrc/tvm_ffi_utils.h`
- `flashinfer/concat_ops.py`
- `include/flashinfer/concat_mla.cuh`
- `include/flashinfer/utils.cuh`
- `tests/utils/test_concat_mla.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-3129`
