---
id: pr-flash-attention-2412
repo: Dao-AILab/flash-attention
pr: 2412
title: Feat([FA4][CUTE DSL]) Add head_dim=256 support (forward + backward)
author: wangsiyu
date: '2026-04-23'
url: https://github.com/Dao-AILab/flash-attention/pull/2412
source_category: upstream-code
architectures:
- sm100
tags:
- FA3
- FA4
- attention_kv
- benchmark_test
- blackwell
- compiler_runtime
- cute
- cute dsl
- head_dim
- scheduler_autotune
- sm100
- softmax
- varlen
techniques:
- async-copy
- autotuning
- tile-scheduling
hardware_features:
- blackwell
- tma
kernel_types:
- attention
- compiler
- scheduler
- softmax
languages:
- cuda-cpp
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: 27b4eb9822d35a6ab3a34e4d5335573e7212fbc5
inclusion_reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.
changed_paths:
- flash_attn/cute/flash_bwd_preprocess.py
- flash_attn/cute/interface.py
- flash_attn/cute/mask.py
- flash_attn/cute/sm100_hd256_2cta_fmha_backward.py
- flash_attn/cute/sm100_hd256_2cta_fmha_backward_dkdvkernel.py
- flash_attn/cute/sm100_hd256_2cta_fmha_backward_dqkernel.py
- flash_attn/cute/sm100_hd256_2cta_fmha_forward.py
- flash_attn/cute/tile_scheduler.py
- flash_attn/cute/utils.py
- tests/cute/test_flash_attn.py
- tests/cute/test_flash_attn_varlen.py
artifact_dir: evidence/pull-bundles/flash-attention/gh-2412
head_ref_oid: c1e5e8c9cacdb1a69cec8d1ce62dd59035cecc76
---
# Feat([FA4][CUTE DSL]) Add head_dim=256 support (forward + backward)

- Repository: `Dao-AILab/flash-attention`
- PR: [2412](https://github.com/Dao-AILab/flash-attention/pull/2412)
- Decision: `Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.`
- Reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.

## Changed Paths

- `flash_attn/cute/flash_bwd_preprocess.py`
- `flash_attn/cute/interface.py`
- `flash_attn/cute/mask.py`
- `flash_attn/cute/sm100_hd256_2cta_fmha_backward.py`
- `flash_attn/cute/sm100_hd256_2cta_fmha_backward_dkdvkernel.py`
- `flash_attn/cute/sm100_hd256_2cta_fmha_backward_dqkernel.py`
- `flash_attn/cute/sm100_hd256_2cta_fmha_forward.py`
- `flash_attn/cute/tile_scheduler.py`
- `flash_attn/cute/utils.py`
- `tests/cute/test_flash_attn.py`
- `tests/cute/test_flash_attn_varlen.py`

## Evidence Bundle

`evidence/pull-bundles/flash-attention/gh-2412`
