---
id: pr-tensorrt-llm-13052
repo: NVIDIA/TensorRT-LLM
pr: 13052
title: '[#12716][feat] Fused cross-head QK Norm + RoPE kernel for WAN'
author: anikaj-eng
date: '2026-05-13'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/13052
source_category: upstream-code
architectures:
- sm100
tags:
- attention kernel
- attention_kv
- benchmark kernel
- benchmark_test
- cuda kernel
- norm_elementwise
techniques:
- kernel-fusion
hardware_features: []
kernel_types:
- attention
- norm
- reduction
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: f7fadf1b84a1bbda53d0900ec6eff39980384fff
inclusion_reason: Mine shape sets, tolerance rules, warmup logic, and profile commands before mutating code.
changed_paths:
- benchmarks/bench_fused_dit_cross_head_qk_norm_rope.py
- cpp/tensorrt_llm/kernels/fusedDiTQKNormRopeKernel.cu
- cpp/tensorrt_llm/kernels/fusedDiTQKNormRopeKernel.h
- cpp/tensorrt_llm/thop/fusedDiTQKNormRopeOp.cpp
- tensorrt_llm/_torch/compilation/utils.py
- tensorrt_llm/_torch/visual_gen/models/wan/transformer_wan.py
- tensorrt_llm/_torch/visual_gen/modules/attention.py
- tests/unittest/_torch/thop/parallel_hw_agnostic/test_fused_dit_qk_norm_rope.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-13052
head_ref_oid: 24fc023e1010a801ce0af148cb7873d400bc8c2a
---
# [#12716][feat] Fused cross-head QK Norm + RoPE kernel for WAN

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [13052](https://github.com/NVIDIA/TensorRT-LLM/pull/13052)
- Decision: `Mine shape sets, tolerance rules, warmup logic, and profile commands before mutating code.`
- Reason: Mine shape sets, tolerance rules, warmup logic, and profile commands before mutating code.

## Changed Paths

- `benchmarks/bench_fused_dit_cross_head_qk_norm_rope.py`
- `cpp/tensorrt_llm/kernels/fusedDiTQKNormRopeKernel.cu`
- `cpp/tensorrt_llm/kernels/fusedDiTQKNormRopeKernel.h`
- `cpp/tensorrt_llm/thop/fusedDiTQKNormRopeOp.cpp`
- `tensorrt_llm/_torch/compilation/utils.py`
- `tensorrt_llm/_torch/visual_gen/models/wan/transformer_wan.py`
- `tensorrt_llm/_torch/visual_gen/modules/attention.py`
- `tests/unittest/_torch/thop/parallel_hw_agnostic/test_fused_dit_qk_norm_rope.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-13052`
