---
id: pr-flash-attention-2109
repo: Dao-AILab/flash-attention
pr: 2109
title: '[Cute,Fwd,Sm100] fp8 e4m3 and e5m2 support'
author: dcw02
date: '2026-04-17'
url: https://github.com/Dao-AILab/flash-attention/pull/2109
source_category: upstream-code
architectures:
- sm100
- sm90
tags:
- FA3
- FA4
- arch_pipeline
- attention_kv
- benchmark_test
- blackwell
- compiler_runtime
- cute
- cute dsl
- fp8
- gemm_quant
- hopper
- sm100
- softmax
techniques:
- async-copy
- fine-grained-quantization
- pipeline-stages
hardware_features:
- blackwell
- fp8
- hopper
- tma
kernel_types:
- attention
- compiler
- gemm
- runtime
- softmax
languages:
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: b65ae6b175f2438de55601695b6a21971fc5e429
inclusion_reason: 'Extract the architecture assumption first: SM target, TMA/WGMMA/tcgen path, cluster use, and pipeline stages.'
changed_paths:
- flash_attn/cute/benchmark_flash_attention_fp8.py
- flash_attn/cute/blackwell_helpers.py
- flash_attn/cute/block_sparse_utils.py
- flash_attn/cute/cute_dsl_utils.py
- flash_attn/cute/flash_fwd_sm100.py
- flash_attn/cute/interface.py
- flash_attn/cute/mma_sm100_desc.py
- flash_attn/cute/softmax.py
artifact_dir: evidence/pull-bundles/flash-attention/gh-2109
head_ref_oid: 52d62009ef8f19f3ad9f28f6a91ee457c58a0ce7
---
# [Cute,Fwd,Sm100] fp8 e4m3 and e5m2 support

- Repository: `Dao-AILab/flash-attention`
- PR: [2109](https://github.com/Dao-AILab/flash-attention/pull/2109)
- Decision: `Extract the architecture assumption first: SM target, TMA/WGMMA/tcgen path, cluster use, and pipeline stages.`
- Reason: Extract the architecture assumption first: SM target, TMA/WGMMA/tcgen path, cluster use, and pipeline stages.

## Changed Paths

- `flash_attn/cute/benchmark_flash_attention_fp8.py`
- `flash_attn/cute/blackwell_helpers.py`
- `flash_attn/cute/block_sparse_utils.py`
- `flash_attn/cute/cute_dsl_utils.py`
- `flash_attn/cute/flash_fwd_sm100.py`
- `flash_attn/cute/interface.py`
- `flash_attn/cute/mma_sm100_desc.py`
- `flash_attn/cute/softmax.py`

## Evidence Bundle

`evidence/pull-bundles/flash-attention/gh-2109`
