---
id: pr-tensorrt-llm-9852
repo: NVIDIA/TensorRT-LLM
pr: 9852
title: '[None][feat] Fused kernels (qknormrope + moe routing) and two-model MTP support for glm4moe'
author: nvxuanyuc
date: '2025-12-14'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/9852
source_category: upstream-code
architectures:
- sm100
tags:
- attention_kv
- benchmark_test
- moe_routing
- norm_elementwise
- trtllmgen
techniques:
- kernel-fusion
hardware_features: []
kernel_types:
- attention
- moe
- norm
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: a5a37227d669cabddb074089143334d9bbb69627
inclusion_reason: Separate routing, permutation, top-k, and expert GEMM costs; keep tail-token and expert-layout tests attached to the idea.
changed_paths:
- cpp/tensorrt_llm/kernels/fusedQKNormRopeKernel.cu
- cpp/tensorrt_llm/kernels/fusedQKNormRopeKernel.h
- cpp/tensorrt_llm/thop/fusedQKNormRopeOp.cpp
- tensorrt_llm/_torch/models/modeling_auto.py
- tensorrt_llm/_torch/models/modeling_glm.py
- tensorrt_llm/_torch/models/modeling_speculative.py
- tensorrt_llm/_torch/modules/qk_norm_attention.py
- tests/integration/defs/accuracy/references/gsm8k.yaml
- tests/integration/defs/accuracy/test_llm_api_pytorch.py
- tests/integration/test_lists/qa/llm_function_core.txt
- tests/unittest/_torch/thop/parallel/test_fused_qk_norm_rope.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-9852
head_ref_oid: 1c8ff8b1274b36948b2db3918834cf5803adbadd
---
# [None][feat] Fused kernels (qknormrope + moe routing) and two-model MTP support for glm4moe

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [9852](https://github.com/NVIDIA/TensorRT-LLM/pull/9852)
- Decision: `Separate routing, permutation, top-k, and expert GEMM costs; keep tail-token and expert-layout tests attached to the idea.`
- Reason: Separate routing, permutation, top-k, and expert GEMM costs; keep tail-token and expert-layout tests attached to the idea.

## Changed Paths

- `cpp/tensorrt_llm/kernels/fusedQKNormRopeKernel.cu`
- `cpp/tensorrt_llm/kernels/fusedQKNormRopeKernel.h`
- `cpp/tensorrt_llm/thop/fusedQKNormRopeOp.cpp`
- `tensorrt_llm/_torch/models/modeling_auto.py`
- `tensorrt_llm/_torch/models/modeling_glm.py`
- `tensorrt_llm/_torch/models/modeling_speculative.py`
- `tensorrt_llm/_torch/modules/qk_norm_attention.py`
- `tests/integration/defs/accuracy/references/gsm8k.yaml`
- `tests/integration/defs/accuracy/test_llm_api_pytorch.py`
- `tests/integration/test_lists/qa/llm_function_core.txt`
- `tests/unittest/_torch/thop/parallel/test_fused_qk_norm_rope.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-9852`
