---
id: pr-flashinfer-2446
repo: flashinfer-ai/flashinfer
pr: 2446
title: 'feat: Add TRTLLM fmha_v2 library for SM90 attention with Skip-Softmax '
author: jimmyzho
date: '2026-03-09'
url: https://github.com/flashinfer-ai/flashinfer/pull/2446
source_category: upstream-code
architectures:
- sm100
- sm90
tags:
- attention
- epilogue-fusion
- flash-attention
- fp8
- kernel-fusion
- prefill
- tma
techniques:
- epilogue-fusion
- kernel-fusion
hardware_features:
- fp8
- tma
kernel_types:
- attention
- flash-attention
- prefill
languages:
- cuda-cpp
captured_at: '2026-04-27'
status: merged
merge_sha: c43bc92888cdeaa3d209557f16a6b7f6c315c2ff
inclusion_reason: kernel file changes
changed_paths:
- csrc/fmha_v2/convert.cu
- csrc/fmha_v2/fmha/gmem_tile_o_packed.h
- csrc/fmha_v2/fmha/gmem_tile_ps.h
- csrc/fmha_v2/fmha/hopper/gmem_tile_o_packed.h
- csrc/fmha_v2/fmha/kernel_traits.h
- csrc/fmha_v2/fmha/warpspec/compute.h
- csrc/fmha_v2/fmha/warpspec/dma.h
- csrc/fmha_v2/fmha/warpspec/epilogue.h
- csrc/fmha_v2/fmha/warpspec/kernel_traits.h
- csrc/fmha_v2/fused_multihead_attention.cpp
- csrc/fmha_v2/fused_multihead_attention.h
- csrc/fmha_v2/fused_multihead_attention_demo_bert_params.h
- csrc/fmha_v2/fused_multihead_cross_attention.cpp
- csrc/fmha_v2/softmax_bf16.cu
- csrc/fmha_v2/softmax_fp16.cu
- csrc/fmha_v2/softmax_fp32.cu
- csrc/fmha_v2/softmax_fp8.cu
- csrc/fmha_v2/softmax_impl.h
- csrc/fmha_v2/softmax_int8.cu
- csrc/fmha_v2/templates/fa_kernel.jinja
- csrc/fmha_v2/templates/kernel.jinja
- csrc/fmha_v2/templates/kernel_hopper.jinja
- csrc/fmha_v2/templates/kernel_hopper_ws.jinja
- csrc/fmha_v2_jit_binding.cu
- csrc/fmha_v2_run.cu
- flashinfer/__init__.py
- flashinfer/jit/__init__.py
- flashinfer/jit/attention/__init__.py
- flashinfer/jit/attention/fmha_v2/fmha_library.py
- flashinfer/jit/attention/fmha_v2/generator_utils.py
- flashinfer/jit/attention/fmha_v2/utils.py
- flashinfer/jit/attention/modules.py
- flashinfer/prefill.py
- tests/attention/test_fmha_v2_prefill.py
- tests/attention/test_fmha_v2_prefill_deepseek.py
head_ref_oid: 4cde9770dc9da1918ef2d724dc6a70878765264c
artifact_dir: evidence/pull-bundles/flashinfer/gh-2446
---
# feat: Add TRTLLM fmha_v2 library for SM90 attention with Skip-Softmax 

- Repository: `flashinfer-ai/flashinfer`
- PR: [2446](https://github.com/flashinfer-ai/flashinfer/pull/2446)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `csrc/fmha_v2/convert.cu`
- `csrc/fmha_v2/fmha/gmem_tile_o_packed.h`
- `csrc/fmha_v2/fmha/gmem_tile_ps.h`
- `csrc/fmha_v2/fmha/hopper/gmem_tile_o_packed.h`
- `csrc/fmha_v2/fmha/kernel_traits.h`
- `csrc/fmha_v2/fmha/warpspec/compute.h`
- `csrc/fmha_v2/fmha/warpspec/dma.h`
- `csrc/fmha_v2/fmha/warpspec/epilogue.h`
- `csrc/fmha_v2/fmha/warpspec/kernel_traits.h`
- `csrc/fmha_v2/fused_multihead_attention.cpp`
- `csrc/fmha_v2/fused_multihead_attention.h`
- `csrc/fmha_v2/fused_multihead_attention_demo_bert_params.h`
- `csrc/fmha_v2/fused_multihead_cross_attention.cpp`
- `csrc/fmha_v2/softmax_bf16.cu`
- `csrc/fmha_v2/softmax_fp16.cu`
- `csrc/fmha_v2/softmax_fp32.cu`
- `csrc/fmha_v2/softmax_fp8.cu`
- `csrc/fmha_v2/softmax_impl.h`
- `csrc/fmha_v2/softmax_int8.cu`
- `csrc/fmha_v2/templates/fa_kernel.jinja`
- `csrc/fmha_v2/templates/kernel.jinja`
- `csrc/fmha_v2/templates/kernel_hopper.jinja`
- `csrc/fmha_v2/templates/kernel_hopper_ws.jinja`
- `csrc/fmha_v2_jit_binding.cu`
- `csrc/fmha_v2_run.cu`
- `flashinfer/__init__.py`
- `flashinfer/jit/__init__.py`
- `flashinfer/jit/attention/__init__.py`
- `flashinfer/jit/attention/fmha_v2/fmha_library.py`
- `flashinfer/jit/attention/fmha_v2/generator_utils.py`
- `flashinfer/jit/attention/fmha_v2/utils.py`
- `flashinfer/jit/attention/modules.py`
- `flashinfer/prefill.py`
- `tests/attention/test_fmha_v2_prefill.py`
- `tests/attention/test_fmha_v2_prefill_deepseek.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-2446`
