---
id: pr-tensorrt-llm-11869
repo: NVIDIA/TensorRT-LLM
pr: 11869
title: '[None][feat] Add fused DiT QK Norm + RoPE CUDA kernel for FLUX'
author: karljang
date: '2026-04-04'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/11869
source_category: upstream-code
architectures:
- sm100
tags:
- attention kernel
- attention_kv
- benchmark_test
- cuda kernel
- norm_elementwise
techniques:
- kernel-fusion
hardware_features: []
kernel_types:
- attention
- norm
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 8f76476a4344070e67e2b375f4e069a6a7449668
inclusion_reason: Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.
changed_paths:
- cpp/include/tensorrt_llm/common/cudaUtils.h
- cpp/tensorrt_llm/kernels/fusedDiTQKNormRopeKernel.cu
- cpp/tensorrt_llm/kernels/fusedDiTQKNormRopeKernel.h
- cpp/tensorrt_llm/kernels/fusedQKNormRopeKernel.cu
- cpp/tensorrt_llm/thop/CMakeLists.txt
- cpp/tensorrt_llm/thop/fusedDiTQKNormRopeOp.cpp
- tensorrt_llm/_torch/compilation/utils.py
- tensorrt_llm/_torch/visual_gen/models/flux/__init__.py
- tensorrt_llm/_torch/visual_gen/models/flux/attention.py
- tensorrt_llm/_torch/visual_gen/models/wan/transformer_wan.py
- tensorrt_llm/_torch/visual_gen/modules/attention.py
- tests/unittest/_torch/thop/parallel/test_fused_dit_qk_norm_rope.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-11869
head_ref_oid: 1c5fef3f86d73b56c5813c3986ff88a41d841636
---
# [None][feat] Add fused DiT QK Norm + RoPE CUDA kernel for FLUX

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [11869](https://github.com/NVIDIA/TensorRT-LLM/pull/11869)
- Decision: `Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.`
- Reason: Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.

## Changed Paths

- `cpp/include/tensorrt_llm/common/cudaUtils.h`
- `cpp/tensorrt_llm/kernels/fusedDiTQKNormRopeKernel.cu`
- `cpp/tensorrt_llm/kernels/fusedDiTQKNormRopeKernel.h`
- `cpp/tensorrt_llm/kernels/fusedQKNormRopeKernel.cu`
- `cpp/tensorrt_llm/thop/CMakeLists.txt`
- `cpp/tensorrt_llm/thop/fusedDiTQKNormRopeOp.cpp`
- `tensorrt_llm/_torch/compilation/utils.py`
- `tensorrt_llm/_torch/visual_gen/models/flux/__init__.py`
- `tensorrt_llm/_torch/visual_gen/models/flux/attention.py`
- `tensorrt_llm/_torch/visual_gen/models/wan/transformer_wan.py`
- `tensorrt_llm/_torch/visual_gen/modules/attention.py`
- `tests/unittest/_torch/thop/parallel/test_fused_dit_qk_norm_rope.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-11869`
