---
id: pr-tensorrt-llm-12507
repo: NVIDIA/TensorRT-LLM
pr: 12507
title: '[#4674][feat] optimize llama8B decode: trtllm silu_mul backend, quant+silu_mul, QKV passthrough to attention'
author: MrGeva
date: '2026-05-07'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/12507
source_category: upstream-code
architectures:
- sm100
tags:
- attention kernel
- benchmark kernel
- triton kernel
techniques:
- fine-grained-quantization
hardware_features: []
kernel_types:
- attention
- decode
languages:
- cuda-cpp
- triton
captured_at: '2026-05-16'
status: merged
merge_sha: f45f52481602db64da2655f7f0f9bbbee62d5301
inclusion_reason: Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.
changed_paths:
- examples/auto_deploy/model_registry/configs/llama3_1_8b.yaml
- tensorrt_llm/_torch/auto_deploy/config/default.yaml
- tensorrt_llm/_torch/auto_deploy/custom_ops/attention/trtllm_attention.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/linear/silu_mul.py
- tensorrt_llm/_torch/auto_deploy/transform/library/fuse_rope_into_trtllm_attention.py
- tensorrt_llm/_torch/auto_deploy/transform/library/fuse_silu_mul.py
- tests/unittest/auto_deploy/singlegpu/transformations/library/test_fuse_silu_mul.py
- tests/unittest/auto_deploy/singlegpu/transformations/library/test_fuse_trtllm_attention_quant_fp8.py
- tests/unittest/auto_deploy/singlegpu/transformations/library/test_gemm_fusion_trtllm.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-12507
head_ref_oid: 826f4eb326845b8aeb84aa6fcb08713e518431e4
---
# [#4674][feat] optimize llama8B decode: trtllm silu_mul backend, quant+silu_mul, QKV passthrough to attention

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [12507](https://github.com/NVIDIA/TensorRT-LLM/pull/12507)
- Decision: `Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.`
- Reason: Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.

## Changed Paths

- `examples/auto_deploy/model_registry/configs/llama3_1_8b.yaml`
- `tensorrt_llm/_torch/auto_deploy/config/default.yaml`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/attention/trtllm_attention.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/linear/silu_mul.py`
- `tensorrt_llm/_torch/auto_deploy/transform/library/fuse_rope_into_trtllm_attention.py`
- `tensorrt_llm/_torch/auto_deploy/transform/library/fuse_silu_mul.py`
- `tests/unittest/auto_deploy/singlegpu/transformations/library/test_fuse_silu_mul.py`
- `tests/unittest/auto_deploy/singlegpu/transformations/library/test_fuse_trtllm_attention_quant_fp8.py`
- `tests/unittest/auto_deploy/singlegpu/transformations/library/test_gemm_fusion_trtllm.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-12507`
