---
id: pr-tensorrt-llm-11273
repo: NVIDIA/TensorRT-LLM
pr: 11273
title: '[None][feat] Optimize super-v3 nvfp4 for better perf'
author: Wanli-Jiang
date: '2026-02-12'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/11273
source_category: upstream-code
architectures:
- sm100
tags:
- benchmark_test
- gemm_quant
- memory_primitives
- norm_elementwise
- sm100
techniques:
- cache-policy
- fine-grained-quantization
- kernel-fusion
hardware_features:
- blackwell
- fp4
- nvfp4
kernel_types:
- gemm
- memory
- norm
- scan
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 421eb9e39c850b89d0c5f266752695b40c175c39
inclusion_reason: Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.
changed_paths:
- cpp/tensorrt_llm/kernels/causalConv1d/causalConv1d.cu
- cpp/tensorrt_llm/kernels/fusedActivationQuant.cu
- cpp/tensorrt_llm/kernels/fusedActivationQuant.h
- cpp/tensorrt_llm/kernels/fusedLayernormKernels/layernorm_param.h
- cpp/tensorrt_llm/kernels/fusedLayernormKernels/low_latency_layernorm.cuh
- cpp/tensorrt_llm/kernels/fusedLayernormKernels/ws_layernorm.cuh
- cpp/tensorrt_llm/kernels/fusedLayernormKernels/ws_layernorm.h
- cpp/tensorrt_llm/kernels/fusedLayernormKernels/ws_layernorm_fp4_traits.cu
- cpp/tensorrt_llm/thop/CMakeLists.txt
- cpp/tensorrt_llm/thop/fusedActivationQuant.cpp
- cpp/tensorrt_llm/thop/fusedAddRMSNormQuant.cpp
- tensorrt_llm/_torch/custom_ops/cpp_custom_ops.py
- tensorrt_llm/_torch/models/modeling_nemotron_h.py
- tensorrt_llm/_torch/modules/mamba/fuse_elementwise_ops.py
- tensorrt_llm/_torch/modules/mamba/mamba2_metadata.py
- tensorrt_llm/_torch/modules/mamba/mamba2_mixer.py
- tensorrt_llm/_torch/modules/mamba/ssd_chunk_scan.py
- tensorrt_llm/_torch/modules/rms_norm.py
- tests/unittest/_torch/modules/mamba/test_causal_conv1d.py
- tests/unittest/_torch/modules/mamba/test_fuse_elementwise_ops.py
- tests/unittest/_torch/modules/mamba/test_mamba2_metadata.py
- tests/unittest/_torch/modules/test_fused_activation_quant.py
- tests/unittest/_torch/modules/test_fused_add_rms_norm_quant.py
- tensorrt_llm/tools/layer_wise_benchmarks/runner.py
- tensorrt_llm/_torch/modules/mamba/ssd_chunk_state.py
- tensorrt_llm/_torch/modules/mlp.py
- tensorrt_llm/_torch/pyexecutor/mamba_cache_manager.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-11273
head_ref_oid: 66f64d0a9ba3ec19362fb220238dd3ebbfac8568
---
# [None][feat] Optimize super-v3 nvfp4 for better perf

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [11273](https://github.com/NVIDIA/TensorRT-LLM/pull/11273)
- Decision: `Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.`
- Reason: Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.

## Changed Paths

- `cpp/tensorrt_llm/kernels/causalConv1d/causalConv1d.cu`
- `cpp/tensorrt_llm/kernels/fusedActivationQuant.cu`
- `cpp/tensorrt_llm/kernels/fusedActivationQuant.h`
- `cpp/tensorrt_llm/kernels/fusedLayernormKernels/layernorm_param.h`
- `cpp/tensorrt_llm/kernels/fusedLayernormKernels/low_latency_layernorm.cuh`
- `cpp/tensorrt_llm/kernels/fusedLayernormKernels/ws_layernorm.cuh`
- `cpp/tensorrt_llm/kernels/fusedLayernormKernels/ws_layernorm.h`
- `cpp/tensorrt_llm/kernels/fusedLayernormKernels/ws_layernorm_fp4_traits.cu`
- `cpp/tensorrt_llm/thop/CMakeLists.txt`
- `cpp/tensorrt_llm/thop/fusedActivationQuant.cpp`
- `cpp/tensorrt_llm/thop/fusedAddRMSNormQuant.cpp`
- `tensorrt_llm/_torch/custom_ops/cpp_custom_ops.py`
- `tensorrt_llm/_torch/models/modeling_nemotron_h.py`
- `tensorrt_llm/_torch/modules/mamba/fuse_elementwise_ops.py`
- `tensorrt_llm/_torch/modules/mamba/mamba2_metadata.py`
- `tensorrt_llm/_torch/modules/mamba/mamba2_mixer.py`
- `tensorrt_llm/_torch/modules/mamba/ssd_chunk_scan.py`
- `tensorrt_llm/_torch/modules/rms_norm.py`
- `tests/unittest/_torch/modules/mamba/test_causal_conv1d.py`
- `tests/unittest/_torch/modules/mamba/test_fuse_elementwise_ops.py`
- `tests/unittest/_torch/modules/mamba/test_mamba2_metadata.py`
- `tests/unittest/_torch/modules/test_fused_activation_quant.py`
- `tests/unittest/_torch/modules/test_fused_add_rms_norm_quant.py`
- `tensorrt_llm/tools/layer_wise_benchmarks/runner.py`
- `tensorrt_llm/_torch/modules/mamba/ssd_chunk_state.py`
- `tensorrt_llm/_torch/modules/mlp.py`
- `tensorrt_llm/_torch/pyexecutor/mamba_cache_manager.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-11273`
