---
id: pr-sglang-24696
repo: sgl-project/sglang
pr: 24696
title: '[Gemma4] Optimize Gemm4 with fused Q/K/V RMSNorm + per-expert FP8 ckpt loader'
author: yuan-luo
date: '2026-05-10'
url: https://github.com/sgl-project/sglang/pull/24696
source_category: upstream-code
architectures:
- sm100
tags:
- attention kernel
- fp8
- fused moe
- fused norm
- gemm_quant
- moe kernel
- moe_routing
- norm_elementwise
techniques:
- fine-grained-quantization
- kernel-fusion
hardware_features:
- fp8
kernel_types:
- attention
- gemm
- moe
- norm
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: d3fd91ed9726cc87d3f2e67e758202d3982cde32
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- python/sglang/srt/layers/gemma4_fused_ops.py
- python/sglang/srt/models/gemma4_causal.py
- python/sglang/srt/models/gemma4_mm.py
- test/registered/models/test_gemma4_fp8_per_expert_loading.py
artifact_dir: evidence/pull-bundles/sglang/gh-24696
head_ref_oid: 2c2b971a0fa0be27245882e4d87c13402f613281
---
# [Gemma4] Optimize Gemm4 with fused Q/K/V RMSNorm + per-expert FP8 ckpt loader

- Repository: `sgl-project/sglang`
- PR: [24696](https://github.com/sgl-project/sglang/pull/24696)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `python/sglang/srt/layers/gemma4_fused_ops.py`
- `python/sglang/srt/models/gemma4_causal.py`
- `python/sglang/srt/models/gemma4_mm.py`
- `test/registered/models/test_gemma4_fp8_per_expert_loading.py`

## Evidence Bundle

`evidence/pull-bundles/sglang/gh-24696`
