---
id: pr-tensorrt-llm-13892
repo: NVIDIA/TensorRT-LLM
pr: 13892
title: '[None][perf] mHC fused_hc kernel optimizations + DS-V4 entry-boundary RMSNorm fold-in'
author: mingyangHao
date: '2026-05-12'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/13892
source_category: upstream-code
architectures:
- sm100
tags:
- attention kernel
- attention_kv
- benchmark_test
- norm_elementwise
techniques:
- kernel-fusion
hardware_features:
- tma
kernel_types:
- attention
- gemm
- moe
- norm
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 2cfd75c8dde2495fd868cf80a047bf7bfc045fd4
inclusion_reason: Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.
changed_paths:
- cpp/tensorrt_llm/kernels/mhcKernels/fused_tf32_pmap_gemm.cuh
- cpp/tensorrt_llm/kernels/mhcKernels/mhcFusedHcKernel.cu
- cpp/tensorrt_llm/kernels/mhcKernels/mhcKernels.cu
- cpp/tensorrt_llm/kernels/mhcKernels/mhcKernels.h
- cpp/tensorrt_llm/kernels/mhcKernels/mhc_fused_fma.cuh
- cpp/tensorrt_llm/thop/mhcOp.cpp
- tensorrt_llm/_torch/models/modeling_deepseekv4.py
- tensorrt_llm/_torch/modules/mhc/hyper_connection.py
- tensorrt_llm/_torch/modules/mhc/mhc_cuda.py
- tests/unittest/_torch/modules/test_mhc.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-13892
head_ref_oid: ba5418f7d9ee1a28fafc817f8423be4df9a573ce
---
# [None][perf] mHC fused_hc kernel optimizations + DS-V4 entry-boundary RMSNorm fold-in

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [13892](https://github.com/NVIDIA/TensorRT-LLM/pull/13892)
- Decision: `Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.`
- Reason: Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.

## Changed Paths

- `cpp/tensorrt_llm/kernels/mhcKernels/fused_tf32_pmap_gemm.cuh`
- `cpp/tensorrt_llm/kernels/mhcKernels/mhcFusedHcKernel.cu`
- `cpp/tensorrt_llm/kernels/mhcKernels/mhcKernels.cu`
- `cpp/tensorrt_llm/kernels/mhcKernels/mhcKernels.h`
- `cpp/tensorrt_llm/kernels/mhcKernels/mhc_fused_fma.cuh`
- `cpp/tensorrt_llm/thop/mhcOp.cpp`
- `tensorrt_llm/_torch/models/modeling_deepseekv4.py`
- `tensorrt_llm/_torch/modules/mhc/hyper_connection.py`
- `tensorrt_llm/_torch/modules/mhc/mhc_cuda.py`
- `tests/unittest/_torch/modules/test_mhc.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-13892`
