---
id: pr-tensorrt-llm-12738
repo: NVIDIA/TensorRT-LLM
pr: 12738
title: '[None][feat] Add bf16 trtllm-gen moe support through flashinfer.'
author: nv-guomingz
date: '2026-04-30'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/12738
source_category: upstream-code
architectures:
- sm100
tags:
- attention_kv
- benchmark_test
- cutlass
- gemm_quant
- moe_routing
- routing
- scheduler_autotune
- trtllmgen
techniques:
- autotuning
- fine-grained-quantization
- kernel-fusion
- tile-scheduling
hardware_features: []
kernel_types:
- attention
- gemm
- moe
- scheduler
languages:
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: dbba6b5ef24637705f97bfc93fe0c4123812154b
inclusion_reason: Separate routing, permutation, top-k, and expert GEMM costs; keep tail-token and expert-layout tests attached to the idea.
changed_paths:
- jenkins/L0_Test.groovy
- tensorrt_llm/_torch/modules/fused_moe/configurable_moe.py
- tensorrt_llm/_torch/modules/fused_moe/create_moe.py
- tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py
- tensorrt_llm/_torch/modules/fused_moe/moe_op_backend.py
- tensorrt_llm/_torch/modules/fused_moe/quantization.py
- tests/integration/defs/.test_durations
- tests/integration/defs/accuracy/test_llm_api_pytorch.py
- tests/integration/test_lists/qa/llm_function_core.txt
- tests/integration/test_lists/test-db/l0_b200.yml
- tests/integration/test_lists/test-db/l0_dgx_b200.yml
- tests/integration/test_lists/waives.txt
- tests/unittest/_torch/modules/moe/moe_test_utils.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-12738
head_ref_oid: 160f0eeb7b38a3107267117865e34a10c8fa1b02
---
# [None][feat] Add bf16 trtllm-gen moe support through flashinfer.

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [12738](https://github.com/NVIDIA/TensorRT-LLM/pull/12738)
- Decision: `Separate routing, permutation, top-k, and expert GEMM costs; keep tail-token and expert-layout tests attached to the idea.`
- Reason: Separate routing, permutation, top-k, and expert GEMM costs; keep tail-token and expert-layout tests attached to the idea.

## Changed Paths

- `jenkins/L0_Test.groovy`
- `tensorrt_llm/_torch/modules/fused_moe/configurable_moe.py`
- `tensorrt_llm/_torch/modules/fused_moe/create_moe.py`
- `tensorrt_llm/_torch/modules/fused_moe/fused_moe_trtllm_gen.py`
- `tensorrt_llm/_torch/modules/fused_moe/moe_op_backend.py`
- `tensorrt_llm/_torch/modules/fused_moe/quantization.py`
- `tests/integration/defs/.test_durations`
- `tests/integration/defs/accuracy/test_llm_api_pytorch.py`
- `tests/integration/test_lists/qa/llm_function_core.txt`
- `tests/integration/test_lists/test-db/l0_b200.yml`
- `tests/integration/test_lists/test-db/l0_dgx_b200.yml`
- `tests/integration/test_lists/waives.txt`
- `tests/unittest/_torch/modules/moe/moe_test_utils.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-12738`
