---
id: pr-flashinfer-2142
repo: flashinfer-ai/flashinfer
pr: 2142
title: 'feat: TRTLLM FMHAv2 backend for ctx attention'
author: jimmyzho
date: '2025-11-28'
url: https://github.com/flashinfer-ai/flashinfer/pull/2142
source_category: upstream-code
architectures:
- sm100
- sm90
tags:
- attention
- epilogue-fusion
- flash-attention
- fp8
- gemm
- kernel-fusion
- prefill
- tma
techniques:
- epilogue-fusion
- kernel-fusion
hardware_features:
- fp8
- tma
kernel_types:
- attention
- flash-attention
- gemm
- prefill
languages:
- cuda-cpp
captured_at: '2026-04-27'
status: merged
merge_sha: b14408b20a19ef2d17c092c29384c98efc4e8a1f
inclusion_reason: kernel file changes
changed_paths:
- csrc/fmha_v2/convert.cu
- csrc/fmha_v2/fmha/alibi_params.h
- csrc/fmha_v2/fmha/fragment.h
- csrc/fmha_v2/fmha/gemm.h
- csrc/fmha_v2/fmha/gmem_tile_o.h
- csrc/fmha_v2/fmha/gmem_tile_o_packed.h
- csrc/fmha_v2/fmha/gmem_tile_ps.h
- csrc/fmha_v2/fmha/gmem_tile_qkv.h
- csrc/fmha_v2/fmha/gmem_tile_qkv_packed.h
- csrc/fmha_v2/fmha/hopper/arrive_wait.h
- csrc/fmha_v2/fmha/hopper/compute_tile.h
- csrc/fmha_v2/fmha/hopper/fragment.h
- csrc/fmha_v2/fmha/hopper/gmem_tile_o_packed.h
- csrc/fmha_v2/fmha/hopper/gmem_tile_qkv_packed.h
- csrc/fmha_v2/fmha/hopper/gmma_descriptor.h
- csrc/fmha_v2/fmha/hopper/kernel_traits.h
- csrc/fmha_v2/fmha/hopper/smem_tile.h
- csrc/fmha_v2/fmha/hopper/smem_tile_o.h
- csrc/fmha_v2/fmha/hopper/tma_descriptor.h
- csrc/fmha_v2/fmha/hopper/tma_types.h
- csrc/fmha_v2/fmha/hopper/utils_gmma.h
- csrc/fmha_v2/fmha/hopper/utils_hgmma.h
- csrc/fmha_v2/fmha/hopper/utils_hgmma_bf16.h
- csrc/fmha_v2/fmha/hopper/utils_igmma.h
- csrc/fmha_v2/fmha/hopper/utils_qgmma.h
- csrc/fmha_v2/fmha/hopper/utils_tma.h
- csrc/fmha_v2/fmha/hopper/utils_warpgroup.h
- csrc/fmha_v2/fmha/kernel_traits.h
- csrc/fmha_v2/fmha/mask.h
- csrc/fmha_v2/fmha/numeric_types.h
- csrc/fmha_v2/fmha/paged_kv_cache.h
- csrc/fmha_v2/fmha/smem_tile.h
- csrc/fmha_v2/fmha/smem_tile_o.h
- csrc/fmha_v2/fmha/smem_tile_qkv.h
- csrc/fmha_v2/fmha/smem_tile_v.h
- csrc/fmha_v2/fmha/softmax.h
- csrc/fmha_v2/fmha/traits.h
- csrc/fmha_v2/fmha/utils.h
- csrc/fmha_v2/fmha/warpspec/circular_buffer.h
- csrc/fmha_v2/fmha/warpspec/compute.h
- csrc/fmha_v2/fmha/warpspec/dma.h
- csrc/fmha_v2/fmha/warpspec/epilogue.h
- csrc/fmha_v2/fmha/warpspec/kernel_traits.h
- csrc/fmha_v2/fused_multihead_attention.cpp
- csrc/fmha_v2/fused_multihead_attention.h
- csrc/fmha_v2/fused_multihead_attention_demo_bert_params.h
- csrc/fmha_v2/fused_multihead_attention_kernel.h
- csrc/fmha_v2/fused_multihead_attention_kernel_1xN.h
- csrc/fmha_v2/fused_multihead_attention_kernel_1xN_multi_cta.h
- csrc/fmha_v2/fused_multihead_attention_kernel_1xN_noloop.h
- csrc/fmha_v2/fused_multihead_attention_kernel_2x2.h
- csrc/fmha_v2/fused_multihead_attention_kernel_4x1_hopper.h
- csrc/fmha_v2/fused_multihead_attention_kernel_4x1_hopper_noloop.h
- csrc/fmha_v2/fused_multihead_attention_kernel_4xN_hopper.h
- csrc/fmha_v2/fused_multihead_attention_kernel_4xN_hopper_noloop.h
- csrc/fmha_v2/fused_multihead_attention_utils.h
- csrc/fmha_v2/fused_multihead_cross_attention.cpp
- csrc/fmha_v2/fused_multihead_cross_attention.h
- csrc/fmha_v2/fused_multihead_cross_attention_kernel_1xN.h
- csrc/fmha_v2/fused_multihead_cross_attention_kernel_1xN_noloop.h
- csrc/fmha_v2/fused_multihead_flash_attention_kernel.h
- csrc/fmha_v2/fused_multihead_flash_attention_kernel_noloop.h
- csrc/fmha_v2/fused_multihead_flash_attention_kernel_noloop_tiled.h
- csrc/fmha_v2/softmax_bf16.cu
- csrc/fmha_v2/softmax_fp16.cu
- csrc/fmha_v2/softmax_fp32.cu
- csrc/fmha_v2/softmax_fp8.cu
- csrc/fmha_v2/softmax_impl.h
- csrc/fmha_v2/softmax_int8.cu
- csrc/trtllm_fmha_v2_binding.cu
- flashinfer/jit/__init__.py
- flashinfer/jit/attention/__init__.py
- flashinfer/jit/attention/fmha_v2/generate_kernels.py
- flashinfer/jit/attention/fmha_v2/generator_utils.py
- flashinfer/jit/attention/modules.py
- flashinfer/prefill.py
- tests/attention/test_fmha_v2_prefill_deepseek.py
head_ref_oid: ec013c1ed635c2458b3d303a92d54b383f47633e
artifact_dir: evidence/pull-bundles/flashinfer/gh-2142
---
# feat: TRTLLM FMHAv2 backend for ctx attention

- Repository: `flashinfer-ai/flashinfer`
- PR: [2142](https://github.com/flashinfer-ai/flashinfer/pull/2142)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `csrc/fmha_v2/convert.cu`
- `csrc/fmha_v2/fmha/alibi_params.h`
- `csrc/fmha_v2/fmha/fragment.h`
- `csrc/fmha_v2/fmha/gemm.h`
- `csrc/fmha_v2/fmha/gmem_tile_o.h`
- `csrc/fmha_v2/fmha/gmem_tile_o_packed.h`
- `csrc/fmha_v2/fmha/gmem_tile_ps.h`
- `csrc/fmha_v2/fmha/gmem_tile_qkv.h`
- `csrc/fmha_v2/fmha/gmem_tile_qkv_packed.h`
- `csrc/fmha_v2/fmha/hopper/arrive_wait.h`
- `csrc/fmha_v2/fmha/hopper/compute_tile.h`
- `csrc/fmha_v2/fmha/hopper/fragment.h`
- `csrc/fmha_v2/fmha/hopper/gmem_tile_o_packed.h`
- `csrc/fmha_v2/fmha/hopper/gmem_tile_qkv_packed.h`
- `csrc/fmha_v2/fmha/hopper/gmma_descriptor.h`
- `csrc/fmha_v2/fmha/hopper/kernel_traits.h`
- `csrc/fmha_v2/fmha/hopper/smem_tile.h`
- `csrc/fmha_v2/fmha/hopper/smem_tile_o.h`
- `csrc/fmha_v2/fmha/hopper/tma_descriptor.h`
- `csrc/fmha_v2/fmha/hopper/tma_types.h`
- `csrc/fmha_v2/fmha/hopper/utils_gmma.h`
- `csrc/fmha_v2/fmha/hopper/utils_hgmma.h`
- `csrc/fmha_v2/fmha/hopper/utils_hgmma_bf16.h`
- `csrc/fmha_v2/fmha/hopper/utils_igmma.h`
- `csrc/fmha_v2/fmha/hopper/utils_qgmma.h`
- `csrc/fmha_v2/fmha/hopper/utils_tma.h`
- `csrc/fmha_v2/fmha/hopper/utils_warpgroup.h`
- `csrc/fmha_v2/fmha/kernel_traits.h`
- `csrc/fmha_v2/fmha/mask.h`
- `csrc/fmha_v2/fmha/numeric_types.h`
- `csrc/fmha_v2/fmha/paged_kv_cache.h`
- `csrc/fmha_v2/fmha/smem_tile.h`
- `csrc/fmha_v2/fmha/smem_tile_o.h`
- `csrc/fmha_v2/fmha/smem_tile_qkv.h`
- `csrc/fmha_v2/fmha/smem_tile_v.h`
- `csrc/fmha_v2/fmha/softmax.h`
- `csrc/fmha_v2/fmha/traits.h`
- `csrc/fmha_v2/fmha/utils.h`
- `csrc/fmha_v2/fmha/warpspec/circular_buffer.h`
- `csrc/fmha_v2/fmha/warpspec/compute.h`
- `csrc/fmha_v2/fmha/warpspec/dma.h`
- `csrc/fmha_v2/fmha/warpspec/epilogue.h`
- `csrc/fmha_v2/fmha/warpspec/kernel_traits.h`
- `csrc/fmha_v2/fused_multihead_attention.cpp`
- `csrc/fmha_v2/fused_multihead_attention.h`
- `csrc/fmha_v2/fused_multihead_attention_demo_bert_params.h`
- `csrc/fmha_v2/fused_multihead_attention_kernel.h`
- `csrc/fmha_v2/fused_multihead_attention_kernel_1xN.h`
- `csrc/fmha_v2/fused_multihead_attention_kernel_1xN_multi_cta.h`
- `csrc/fmha_v2/fused_multihead_attention_kernel_1xN_noloop.h`
- `csrc/fmha_v2/fused_multihead_attention_kernel_2x2.h`
- `csrc/fmha_v2/fused_multihead_attention_kernel_4x1_hopper.h`
- `csrc/fmha_v2/fused_multihead_attention_kernel_4x1_hopper_noloop.h`
- `csrc/fmha_v2/fused_multihead_attention_kernel_4xN_hopper.h`
- `csrc/fmha_v2/fused_multihead_attention_kernel_4xN_hopper_noloop.h`
- `csrc/fmha_v2/fused_multihead_attention_utils.h`
- `csrc/fmha_v2/fused_multihead_cross_attention.cpp`
- `csrc/fmha_v2/fused_multihead_cross_attention.h`
- `csrc/fmha_v2/fused_multihead_cross_attention_kernel_1xN.h`
- `csrc/fmha_v2/fused_multihead_cross_attention_kernel_1xN_noloop.h`
- `csrc/fmha_v2/fused_multihead_flash_attention_kernel.h`
- `csrc/fmha_v2/fused_multihead_flash_attention_kernel_noloop.h`
- `csrc/fmha_v2/fused_multihead_flash_attention_kernel_noloop_tiled.h`
- `csrc/fmha_v2/softmax_bf16.cu`
- `csrc/fmha_v2/softmax_fp16.cu`
- `csrc/fmha_v2/softmax_fp32.cu`
- `csrc/fmha_v2/softmax_fp8.cu`
- `csrc/fmha_v2/softmax_impl.h`
- `csrc/fmha_v2/softmax_int8.cu`
- `csrc/trtllm_fmha_v2_binding.cu`
- `flashinfer/jit/__init__.py`
- `flashinfer/jit/attention/__init__.py`
- `flashinfer/jit/attention/fmha_v2/generate_kernels.py`
- `flashinfer/jit/attention/fmha_v2/generator_utils.py`
- `flashinfer/jit/attention/modules.py`
- `flashinfer/prefill.py`
- `tests/attention/test_fmha_v2_prefill_deepseek.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-2142`
