---
id: pr-sglang-21321
repo: sgl-project/sglang
pr: 21321
title: '[Kernel] Support FlashInfer TRTLLM-Gen fused MoE for non-gated FP4 & FP8 (Nemotron)'
author: danielafrimi
date: '2026-04-29'
url: https://github.com/sgl-project/sglang/pull/21321
source_category: upstream-code
architectures:
- sm100
tags:
- attention_kv
- cutlass
- flashinfer
- fp4
- gemm_quant
- moe_routing
- nvfp4
techniques:
- fine-grained-quantization
- kernel-fusion
hardware_features:
- fp4
- fp8
- nvfp4
kernel_types:
- attention
- gemm
- moe
languages:
- cuda-cpp
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: 8327270c7263fb9f62e03a1e7106f079aa7216f3
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- python/sglang/srt/layers/moe/flashinfer_trtllm_moe.py
- python/sglang/srt/layers/moe/moe_runner/flashinfer_trtllm.py
- python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8_moe.py
- python/sglang/srt/layers/quantization/fp8.py
- python/sglang/srt/layers/quantization/modelopt_quant.py
- python/sglang/srt/layers/quantization/utils.py
- python/sglang/srt/models/nemotron_h.py
- python/sglang/srt/server_args.py
artifact_dir: evidence/pull-bundles/sglang/gh-21321
head_ref_oid: db5b3de97d6558a16be8ba88f0de0a5af5f41db0
---
# [Kernel] Support FlashInfer TRTLLM-Gen fused MoE for non-gated FP4 & FP8 (Nemotron)

- Repository: `sgl-project/sglang`
- PR: [21321](https://github.com/sgl-project/sglang/pull/21321)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `python/sglang/srt/layers/moe/flashinfer_trtllm_moe.py`
- `python/sglang/srt/layers/moe/moe_runner/flashinfer_trtllm.py`
- `python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8_moe.py`
- `python/sglang/srt/layers/quantization/fp8.py`
- `python/sglang/srt/layers/quantization/modelopt_quant.py`
- `python/sglang/srt/layers/quantization/utils.py`
- `python/sglang/srt/models/nemotron_h.py`
- `python/sglang/srt/server_args.py`

## Evidence Bundle

`evidence/pull-bundles/sglang/gh-21321`
