---
id: pr-tensorrt-llm-13929
repo: NVIDIA/TensorRT-LLM
pr: 13929
title: '[TRTLLM-35237][feat] Add cute dsl FP4 paged MQA logits decode kernel'
author: limin2021
date: '2026-05-14'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/13929
source_category: upstream-code
architectures:
- sm100
tags:
- DSA
- arch_pipeline
- attention kernel
- attention_kv
- benchmark kernel
- compiler_runtime
- cute dsl
- fp4
- gemm_quant
- paged MQA
- sm100
techniques:
- fine-grained-quantization
- pipeline-stages
hardware_features:
- blackwell
- fp4
- fp8
kernel_types:
- attention
- compiler
- decode
- gemm
- runtime
languages:
- cuda-cpp
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: b9c17305a58dc8a44fe4edf109163a110b8d3093
inclusion_reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.
changed_paths:
- tensorrt_llm/_torch/attention_backend/sparse/dsa.py
- tensorrt_llm/_torch/custom_ops/cute_dsl_custom_ops.py
- tensorrt_llm/_torch/cute_dsl_kernels/blackwell/paged_mqa_logits/__init__.py
- tensorrt_llm/_torch/cute_dsl_kernels/blackwell/paged_mqa_logits/fp4_paged_mqa_logits.py
- tensorrt_llm/_torch/cute_dsl_kernels/blackwell/paged_mqa_logits/fp8_paged_mqa_logits.py
- tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/utils.py
- tests/integration/defs/accuracy/test_llm_api_pytorch.py
- tests/integration/test_lists/qa/llm_function_core.txt
- tests/integration/test_lists/test-db/l0_dgx_b200.yml
- tests/scripts/cute_dsl_kernels/paged_mqa_logits/run_fp4.py
- tests/scripts/cute_dsl_kernels/paged_mqa_logits/run_fp8.py
- tests/unittest/_torch/attention/sparse/test_cute_dsl_fp4_paged_mqa_logits.py
- tests/unittest/_torch/attention/sparse/test_cute_dsl_fp8_paged_mqa_logits.py
- tests/unittest/_torch/attention/sparse/test_dsa_indexer.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-13929
head_ref_oid: 33a803d1638fd64482659099ab24b697f2b418b7
---
# [TRTLLM-35237][feat] Add cute dsl FP4 paged MQA logits decode kernel

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [13929](https://github.com/NVIDIA/TensorRT-LLM/pull/13929)
- Decision: `Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.`
- Reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.

## Changed Paths

- `tensorrt_llm/_torch/attention_backend/sparse/dsa.py`
- `tensorrt_llm/_torch/custom_ops/cute_dsl_custom_ops.py`
- `tensorrt_llm/_torch/cute_dsl_kernels/blackwell/paged_mqa_logits/__init__.py`
- `tensorrt_llm/_torch/cute_dsl_kernels/blackwell/paged_mqa_logits/fp4_paged_mqa_logits.py`
- `tensorrt_llm/_torch/cute_dsl_kernels/blackwell/paged_mqa_logits/fp8_paged_mqa_logits.py`
- `tensorrt_llm/_torch/visual_gen/jit_kernels/flash_attention/cute/utils.py`
- `tests/integration/defs/accuracy/test_llm_api_pytorch.py`
- `tests/integration/test_lists/qa/llm_function_core.txt`
- `tests/integration/test_lists/test-db/l0_dgx_b200.yml`
- `tests/scripts/cute_dsl_kernels/paged_mqa_logits/run_fp4.py`
- `tests/scripts/cute_dsl_kernels/paged_mqa_logits/run_fp8.py`
- `tests/unittest/_torch/attention/sparse/test_cute_dsl_fp4_paged_mqa_logits.py`
- `tests/unittest/_torch/attention/sparse/test_cute_dsl_fp8_paged_mqa_logits.py`
- `tests/unittest/_torch/attention/sparse/test_dsa_indexer.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-13929`
