---
id: pr-tensorrt-llm-11697
repo: NVIDIA/TensorRT-LLM
pr: 11697
title: '[TRTLLM-11092][feat] add support for visual gen FA4 attention backend'
author: o-stoner
date: '2026-03-12'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/11697
source_category: upstream-code
architectures:
- sm100
tags:
- arch_pipeline
- attention_kv
- benchmark_test
- compiler_runtime
- sm100
techniques:
- pipeline-stages
hardware_features:
- blackwell
kernel_types:
- attention
- compiler
- runtime
languages:
- cuda-cpp
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: e8242643066019a186266ee0912b3bf35804c868
inclusion_reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.
changed_paths:
- .pre-commit-config.yaml
- LICENSE
- examples/visual_gen/README.md
- examples/visual_gen/visual_gen_flux.py
- examples/visual_gen/visual_gen_wan_i2v.py
- examples/visual_gen/visual_gen_wan_t2v.py
- tensorrt_llm/_torch/visual_gen/attention_backend/__init__.py
- tensorrt_llm/_torch/visual_gen/attention_backend/flash_attn4.py
- tensorrt_llm/_torch/visual_gen/attention_backend/utils.py
- tensorrt_llm/_torch/visual_gen/config.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/__init__.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/__init__.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/.flake8
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/AUTHORS
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/LICENSE
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/README.md
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/__init__.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/ampere_helpers.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/barrier.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/blackwell_helpers.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/block_info.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/block_sparse_utils.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/block_sparsity.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/benchmark.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-11697
head_ref_oid: f1a41c211ae7b6c534fb9f02f28276e259909237
---
# [TRTLLM-11092][feat] add support for visual gen FA4 attention backend

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [11697](https://github.com/NVIDIA/TensorRT-LLM/pull/11697)
- Decision: `Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.`
- Reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.

## Changed Paths

- `.pre-commit-config.yaml`
- `LICENSE`
- `examples/visual_gen/README.md`
- `examples/visual_gen/visual_gen_flux.py`
- `examples/visual_gen/visual_gen_wan_i2v.py`
- `examples/visual_gen/visual_gen_wan_t2v.py`
- `tensorrt_llm/_torch/visual_gen/attention_backend/__init__.py`
- `tensorrt_llm/_torch/visual_gen/attention_backend/flash_attn4.py`
- `tensorrt_llm/_torch/visual_gen/attention_backend/utils.py`
- `tensorrt_llm/_torch/visual_gen/config.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/__init__.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/__init__.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/.flake8`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/AUTHORS`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/LICENSE`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/README.md`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/__init__.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/ampere_helpers.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/barrier.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/blackwell_helpers.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/block_info.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/block_sparse_utils.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/block_sparsity.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/benchmark.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-11697`
