---
id: pr-flashinfer-1396
repo: flashinfer-ai/flashinfer
pr: 1396
title: 'gpt-oss: Add MXFP8 x MXFP4 CUTLASS MOE for SM100 and BF16 x MXFP4 CUTLASS for SM90 + SwigluBias Activation'
author: djmmoss
date: '2025-08-19'
url: https://github.com/flashinfer-ai/flashinfer/pull/1396
source_category: upstream-code
architectures:
- sm100
- sm90
tags:
- fp4
- fp8
- gemm
- kernel-fusion
- moe
- quantization
- tma
- warp-specialization
techniques:
- kernel-fusion
- warp-specialization
hardware_features:
- fp4
- fp8
- tma
kernel_types:
- gemm
- moe
- quantization
languages:
- cuda-cpp
captured_at: '2026-04-27'
status: merged
merge_sha: dd9a333412ab15680affe6a949a60ea911295a2a
inclusion_reason: kernel file changes
changed_paths:
- csrc/fused_moe/cutlass_backend/cutlass_fused_moe_instantiation.cu
- csrc/fused_moe/cutlass_backend/cutlass_fused_moe_kernels.cuh
- csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_ops.cu
- csrc/nv_internal/cpp/kernels/quantization.cu
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/detail/collective/mixed_input_utils.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/builders/sm90_gmma_builder_gated.inl
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/builders/sm90_gmma_builder_interleaved.inl
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/builders/sm90_gmma_builder_mixed_input.inl
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_builder_gated.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_builder_interleaved.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_builder_mixed_input.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_mma_array_mixed_input.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_mma_gated.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_mma_interleaved.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_array_tma_gmma_rs_warpspecialized_mixed_input_.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_gated_tma_gmma_ss_warpspecialized.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_gated_tma_gmma_ss_warpspecialized_fp8.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_interleaved_tma_gmma_rs_warpspecialized_mixed_input.hpp
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm_configs.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int4_gemm_fg_scalebias.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int4_gemm_fg_scaleonly.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int4_gemm_per_col.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int8_gemm_fg_scalebias.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int8_gemm_fg_scaleonly.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int8_gemm_per_col.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/e4m3_int4_gemm_fg_scalebias_bf16_out_bf16.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/e4m3_int4_gemm_fg_scalebias_f16_out_f16.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/e4m3_int4_gemm_fg_scaleonly_bf16_out_bf16.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/e4m3_int4_gemm_fg_scaleonly_f16_out_f16.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/e4m3_int4_gemm_per_col_f16_out_f16.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int4_gemm_fg_scalebias.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int4_gemm_fg_scaleonly.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int4_gemm_per_col.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int8_gemm_fg_scalebias.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int8_gemm_fg_scaleonly.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int8_gemm_per_col.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fpA_intB_gemm.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fpA_intB_gemm_template.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fpA_intB_gemm_template_sm90.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/launchers/fpA_intB_launcher_sm90.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/launchers/fpA_intB_launcher_sm90.inl
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/include/common.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/include/moe_gemm_kernels.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/include/moe_kernels.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/launchers/moe_gemm_tma_ws_mixed_input_launcher.inl
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_kernels_bf16_fp4.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_kernels_fp16_fp4.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_template_dispatch.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_template_dispatch_tma_ws_mixed_dtype.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_tma_warp_specialized_traits.h
- csrc/nv_internal/tensorrt_llm/kernels/quantization.cuh
- csrc/nv_internal/tensorrt_llm/kernels/quantization.h
- csrc/nv_internal/tensorrt_llm/thop/fp4Op.cpp
- csrc/nv_internal/tensorrt_llm/thop/fp4Quantize.cpp
- csrc/nv_internal/tensorrt_llm/thop/fp4Quantize.h
- csrc/nv_internal/tensorrt_llm/thop/fp8Quantize.cpp
- csrc/nv_internal/tensorrt_llm/thop/fp8Quantize.h
- csrc/trtllm_allreduce_fusion.cu
- csrc/trtllm_fused_moe_kernel_launcher.cu
- csrc/trtllm_moe_allreduce_fusion.cu
- flashinfer/__init__.py
- flashinfer/aot.py
- flashinfer/comm/__init__.py
- flashinfer/comm/trtllm_ar.py
- flashinfer/fp4_quantization.py
- flashinfer/fp8_quantization.py
- flashinfer/fused_moe/__init__.py
- flashinfer/fused_moe/core.py
- flashinfer/fused_moe/utils.py
- flashinfer/jit/core.py
- flashinfer/jit/cutlass_gemm/generate_kernels.py
- include/flashinfer/comm/trtllm_allreduce_fusion.cuh
- include/flashinfer/comm/trtllm_moe_allreduce_fusion.cuh
- tests/test_fp4_quantize.py
- tests/test_groupwise_scaled_gemm_mxfp4.py
- tests/test_trtllm_allreduce_fusion.py
- tests/test_trtllm_cutlass_fused_moe.py
- tests/test_trtllm_gen_fused_moe.py
- tests/test_trtllm_moe_allreduce_fusion.py
head_ref_oid: c46b93455a7b2ba9cf89d3e8a9f3bcd107d516db
artifact_dir: evidence/pull-bundles/flashinfer/gh-1396
---
# gpt-oss: Add MXFP8 x MXFP4 CUTLASS MOE for SM100 and BF16 x MXFP4 CUTLASS for SM90 + SwigluBias Activation

- Repository: `flashinfer-ai/flashinfer`
- PR: [1396](https://github.com/flashinfer-ai/flashinfer/pull/1396)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `csrc/fused_moe/cutlass_backend/cutlass_fused_moe_instantiation.cu`
- `csrc/fused_moe/cutlass_backend/cutlass_fused_moe_kernels.cuh`
- `csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_ops.cu`
- `csrc/nv_internal/cpp/kernels/quantization.cu`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/detail/collective/mixed_input_utils.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/builders/sm90_gmma_builder_gated.inl`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/builders/sm90_gmma_builder_interleaved.inl`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/builders/sm90_gmma_builder_mixed_input.inl`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_builder_gated.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_builder_interleaved.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_builder_mixed_input.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_mma_array_mixed_input.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_mma_gated.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_mma_interleaved.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_array_tma_gmma_rs_warpspecialized_mixed_input_.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_gated_tma_gmma_ss_warpspecialized.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_gated_tma_gmma_ss_warpspecialized_fp8.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_interleaved_tma_gmma_rs_warpspecialized_mixed_input.hpp`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm_configs.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int4_gemm_fg_scalebias.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int4_gemm_fg_scaleonly.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int4_gemm_per_col.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int8_gemm_fg_scalebias.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int8_gemm_fg_scaleonly.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/bf16_int8_gemm_per_col.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/e4m3_int4_gemm_fg_scalebias_bf16_out_bf16.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/e4m3_int4_gemm_fg_scalebias_f16_out_f16.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/e4m3_int4_gemm_fg_scaleonly_bf16_out_bf16.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/e4m3_int4_gemm_fg_scaleonly_f16_out_f16.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/e4m3_int4_gemm_per_col_f16_out_f16.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int4_gemm_fg_scalebias.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int4_gemm_fg_scaleonly.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int4_gemm_per_col.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int8_gemm_fg_scalebias.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int8_gemm_fg_scaleonly.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fp16_int8_gemm_per_col.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fpA_intB_gemm.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fpA_intB_gemm_template.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/fpA_intB_gemm_template_sm90.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/launchers/fpA_intB_launcher_sm90.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/fpA_intB_gemm/launchers/fpA_intB_launcher_sm90.inl`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/include/common.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/include/moe_gemm_kernels.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/include/moe_kernels.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/launchers/moe_gemm_tma_ws_mixed_input_launcher.inl`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_kernels_bf16_fp4.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_kernels_fp16_fp4.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_template_dispatch.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_template_dispatch_tma_ws_mixed_dtype.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_tma_warp_specialized_traits.h`
- `csrc/nv_internal/tensorrt_llm/kernels/quantization.cuh`
- `csrc/nv_internal/tensorrt_llm/kernels/quantization.h`
- `csrc/nv_internal/tensorrt_llm/thop/fp4Op.cpp`
- `csrc/nv_internal/tensorrt_llm/thop/fp4Quantize.cpp`
- `csrc/nv_internal/tensorrt_llm/thop/fp4Quantize.h`
- `csrc/nv_internal/tensorrt_llm/thop/fp8Quantize.cpp`
- `csrc/nv_internal/tensorrt_llm/thop/fp8Quantize.h`
- `csrc/trtllm_allreduce_fusion.cu`
- `csrc/trtllm_fused_moe_kernel_launcher.cu`
- `csrc/trtllm_moe_allreduce_fusion.cu`
- `flashinfer/__init__.py`
- `flashinfer/aot.py`
- `flashinfer/comm/__init__.py`
- `flashinfer/comm/trtllm_ar.py`
- `flashinfer/fp4_quantization.py`
- `flashinfer/fp8_quantization.py`
- `flashinfer/fused_moe/__init__.py`
- `flashinfer/fused_moe/core.py`
- `flashinfer/fused_moe/utils.py`
- `flashinfer/jit/core.py`
- `flashinfer/jit/cutlass_gemm/generate_kernels.py`
- `include/flashinfer/comm/trtllm_allreduce_fusion.cuh`
- `include/flashinfer/comm/trtllm_moe_allreduce_fusion.cuh`
- `tests/test_fp4_quantize.py`
- `tests/test_groupwise_scaled_gemm_mxfp4.py`
- `tests/test_trtllm_allreduce_fusion.py`
- `tests/test_trtllm_cutlass_fused_moe.py`
- `tests/test_trtllm_gen_fused_moe.py`
- `tests/test_trtllm_moe_allreduce_fusion.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-1396`
