---
id: pr-sglang-24490
repo: sgl-project/sglang
pr: 24490
title: Port MXFP4 Marlin MoE support to JIT kernel path
author: yhyang201
date: '2026-05-07'
url: https://github.com/sgl-project/sglang/pull/24490
source_category: upstream-code
architectures:
- sm100
- sm90
tags:
- CUDA kernel
- deepgemm
- fp4
- fused moe
- gemm_quant
- hopper
- moe kernel
- mxfp4
- sgl-kernel
techniques:
- fine-grained-quantization
- kernel-fusion
hardware_features:
- fp4
- fp8
- hopper
- mxfp4
kernel_types:
- gemm
- moe
languages:
- cuda-cpp
- triton
captured_at: '2026-05-16'
status: merged
merge_sha: 6efeee89179d1cd4c50c9790765272d2971c1641
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- python/sglang/jit_kernel/csrc/gemm/marlin_moe/marlin_template.h
- python/sglang/srt/layers/moe/fused_moe_triton/fused_marlin_moe.py
- python/sglang/srt/layers/moe/moe_runner/marlin.py
- python/sglang/srt/layers/quantization/fp8.py
- python/sglang/srt/layers/quantization/marlin_utils_fp4.py
- python/sglang/srt/layers/quantization/mxfp4_flashinfer_trtllm_moe.py
- python/sglang/srt/layers/quantization/mxfp4_marlin_moe.py
artifact_dir: evidence/pull-bundles/sglang/gh-24490
head_ref_oid: cca00c66e7880f12bcdc3ca750995acb92c16c44
---
# Port MXFP4 Marlin MoE support to JIT kernel path

- Repository: `sgl-project/sglang`
- PR: [24490](https://github.com/sgl-project/sglang/pull/24490)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `python/sglang/jit_kernel/csrc/gemm/marlin_moe/marlin_template.h`
- `python/sglang/srt/layers/moe/fused_moe_triton/fused_marlin_moe.py`
- `python/sglang/srt/layers/moe/moe_runner/marlin.py`
- `python/sglang/srt/layers/quantization/fp8.py`
- `python/sglang/srt/layers/quantization/marlin_utils_fp4.py`
- `python/sglang/srt/layers/quantization/mxfp4_flashinfer_trtllm_moe.py`
- `python/sglang/srt/layers/quantization/mxfp4_marlin_moe.py`

## Evidence Bundle

`evidence/pull-bundles/sglang/gh-24490`
