---
id: pr-tensorrt-llm-9905
repo: NVIDIA/TensorRT-LLM
pr: 9905
title: '[None][feat] Adding torch ext API for FusedAddRMSNormQuant kernel'
author: JintaoPengCS
date: '2026-01-14'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/9905
source_category: upstream-code
architectures:
- sm90
tags:
- benchmark_test
- gemm_quant
- norm_elementwise
- sm90
techniques:
- fine-grained-quantization
- kernel-fusion
hardware_features:
- fp4
- hopper
kernel_types:
- gemm
- norm
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 211c44b9519962f209138a180da8936f84a4f02a
inclusion_reason: Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.
changed_paths:
- cpp/tensorrt_llm/kernels/fusedLayernormKernels/low_latency_layernorm.cuh
- cpp/tensorrt_llm/kernels/fusedLayernormKernels/ws_layernorm.cuh
- cpp/tensorrt_llm/thop/CMakeLists.txt
- cpp/tensorrt_llm/thop/fusedAddRMSNormQuant.cpp
- tensorrt_llm/_torch/custom_ops/torch_custom_ops.py
- tensorrt_llm/_torch/models/modeling_llama.py
- tensorrt_llm/_torch/modules/rms_norm.py
- tensorrt_llm/mapping.py
- tests/integration/defs/accuracy/test_llm_api_pytorch.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-9905
head_ref_oid: 00daae0d1a2dccf781dcd687579b5de1e0491593
---
# [None][feat] Adding torch ext API for FusedAddRMSNormQuant kernel

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [9905](https://github.com/NVIDIA/TensorRT-LLM/pull/9905)
- Decision: `Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.`
- Reason: Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.

## Changed Paths

- `cpp/tensorrt_llm/kernels/fusedLayernormKernels/low_latency_layernorm.cuh`
- `cpp/tensorrt_llm/kernels/fusedLayernormKernels/ws_layernorm.cuh`
- `cpp/tensorrt_llm/thop/CMakeLists.txt`
- `cpp/tensorrt_llm/thop/fusedAddRMSNormQuant.cpp`
- `tensorrt_llm/_torch/custom_ops/torch_custom_ops.py`
- `tensorrt_llm/_torch/models/modeling_llama.py`
- `tensorrt_llm/_torch/modules/rms_norm.py`
- `tensorrt_llm/mapping.py`
- `tests/integration/defs/accuracy/test_llm_api_pytorch.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-9905`
