---
id: pr-flashinfer-3084
repo: flashinfer-ai/flashinfer
pr: 3084
title: 'perf: optimize MXFP4xBF16 & INT4xFP8 CUTLASS MoE backend for SM90'
author: samuellees
date: '2026-04-23'
url: https://github.com/flashinfer-ai/flashinfer/pull/3084
source_category: upstream-code
architectures:
- sm100
- sm90
tags:
- TMA
- arch_pipeline
- benchmark_test
- cuda kernel
- cutlass
- fp4
- fused moe
- gemm_quant
- hopper
- moe_routing
- mxfp4
- sm90
- topk
techniques:
- async-copy
- fine-grained-quantization
- kernel-fusion
- pipeline-stages
- warp-specialization
hardware_features:
- fp4
- fp8
- hopper
- mxfp4
- tma
kernel_types:
- gemm
- moe
- runtime
languages:
- cuda-cpp
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: d4544921ca2abee84077679988bfc91f69acd5e8
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_binding.cu
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/detail/collective/mixed_input_utils.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_array_tma_gmma_rs_warpspecialized_mixed_input_.hpp
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/launchers/moe_gemm_tma_ws_mixed_input_launcher.inl
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_mixed_utils.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_mixed_utils.h
- docs/api/fused_moe.rst
- flashinfer/fused_moe/__init__.py
- flashinfer/fused_moe/core.py
- flashinfer/jit/fused_moe.py
- tests/moe/test_trtllm_cutlass_fused_moe.py
artifact_dir: evidence/pull-bundles/flashinfer/gh-3084
head_ref_oid: 1a5b2421ba1a3fcd945926e0f7214fd336e87a2a
---
# perf: optimize MXFP4xBF16 & INT4xFP8 CUTLASS MoE backend for SM90

- Repository: `flashinfer-ai/flashinfer`
- PR: [3084](https://github.com/flashinfer-ai/flashinfer/pull/3084)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_binding.cu`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/detail/collective/mixed_input_utils.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_array_tma_gmma_rs_warpspecialized_mixed_input_.hpp`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/launchers/moe_gemm_tma_ws_mixed_input_launcher.inl`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_mixed_utils.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_mixed_utils.h`
- `docs/api/fused_moe.rst`
- `flashinfer/fused_moe/__init__.py`
- `flashinfer/fused_moe/core.py`
- `flashinfer/jit/fused_moe.py`
- `tests/moe/test_trtllm_cutlass_fused_moe.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-3084`
