---
id: pr-vllm-33255
repo: vllm-project/vllm
pr: 33255
title: '[Bugfix] Fix quant RMS norm fusion for quantization with TMA-aligned scales'
author: ElizaWszola
date: '2026-02-18'
url: https://github.com/vllm-project/vllm/pull/33255
source_category: upstream-code
architectures:
- sm90
tags:
- gemm_quant
- hopper
- norm_elementwise
techniques:
- async-copy
- fine-grained-quantization
- kernel-fusion
hardware_features:
- fp8
- hopper
- tma
kernel_types:
- gemm
- norm
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: a88b3be7c4c494c896f4a88ca8e6bfc1083625e0
inclusion_reason: Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.
changed_paths:
- csrc/ops.h
- csrc/quantization/fused_kernels/fused_layernorm_dynamic_per_token_quant.cu
- csrc/quantization/fused_kernels/layernorm_utils.cuh
- csrc/quantization/w8a8/fp8/per_token_group_quant.cu
- csrc/torch_bindings.cpp
- tests/compile/fusions_e2e/test_tp1_quant.py
- tests/kernels/core/test_fused_quant_layernorm.py
- vllm/_custom_ops.py
- vllm/compilation/passes/fusion/matcher_utils.py
- vllm/compilation/passes/fusion/rms_quant_fusion.py
- vllm/model_executor/layers/quantization/utils/fp8_utils.py
- vllm/utils/deep_gemm.py
artifact_dir: evidence/pull-bundles/vllm/gh-33255
head_ref_oid: 19a40c50b73abc75f59fa8c46eada0b223750f52
---
# [Bugfix] Fix quant RMS norm fusion for quantization with TMA-aligned scales

- Repository: `vllm-project/vllm`
- PR: [33255](https://github.com/vllm-project/vllm/pull/33255)
- Decision: `Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.`
- Reason: Look for fusion, vectorization, dtype conversion, and store-traffic reductions before changing math.

## Changed Paths

- `csrc/ops.h`
- `csrc/quantization/fused_kernels/fused_layernorm_dynamic_per_token_quant.cu`
- `csrc/quantization/fused_kernels/layernorm_utils.cuh`
- `csrc/quantization/w8a8/fp8/per_token_group_quant.cu`
- `csrc/torch_bindings.cpp`
- `tests/compile/fusions_e2e/test_tp1_quant.py`
- `tests/kernels/core/test_fused_quant_layernorm.py`
- `vllm/_custom_ops.py`
- `vllm/compilation/passes/fusion/matcher_utils.py`
- `vllm/compilation/passes/fusion/rms_quant_fusion.py`
- `vllm/model_executor/layers/quantization/utils/fp8_utils.py`
- `vllm/utils/deep_gemm.py`

## Evidence Bundle

`evidence/pull-bundles/vllm/gh-33255`
