---
id: pr-sglang-4686
repo: sgl-project/sglang
pr: 4686
title: Fix loading KV quantization scale; Enable modelopt kv cache
author: yundai424
date: '2025-04-08'
url: https://github.com/sgl-project/sglang/pull/4686
source_category: upstream-code
architectures:
- sm100
tags:
- attention
- gemm
- moe
- quantization
techniques: []
hardware_features: []
kernel_types:
- attention
- gemm
- moe
- quantization
languages:
- python
captured_at: '2026-04-27'
status: merged
merge_sha: 2695ab05375456b4a2ac625cca12b95c3b90e5ae
inclusion_reason: kernel file changes
changed_paths:
- python/sglang/srt/configs/model_config.py
- python/sglang/srt/layers/attention/flashattention_backend.py
- python/sglang/srt/layers/attention/flashinfer_backend.py
- python/sglang/srt/layers/quantization/kv_cache.py
- python/sglang/srt/layers/quantization/modelopt_quant.py
- python/sglang/srt/layers/radix_attention.py
- python/sglang/srt/models/baichuan.py
- python/sglang/srt/models/chatglm.py
- python/sglang/srt/models/commandr.py
- python/sglang/srt/models/dbrx.py
- python/sglang/srt/models/deepseek.py
- python/sglang/srt/models/deepseek_v2.py
- python/sglang/srt/models/exaone.py
- python/sglang/srt/models/gemma.py
- python/sglang/srt/models/gemma2.py
- python/sglang/srt/models/gemma3_causal.py
- python/sglang/srt/models/gpt2.py
- python/sglang/srt/models/gpt_bigcode.py
- python/sglang/srt/models/granite.py
- python/sglang/srt/models/grok.py
- python/sglang/srt/models/internlm2.py
- python/sglang/srt/models/llama.py
- python/sglang/srt/models/minicpm.py
- python/sglang/srt/models/minicpm3.py
- python/sglang/srt/models/mixtral.py
- python/sglang/srt/models/mixtral_quant.py
- python/sglang/srt/models/mllama.py
- python/sglang/srt/models/olmo.py
- python/sglang/srt/models/olmo2.py
- python/sglang/srt/models/olmoe.py
- python/sglang/srt/models/phi3_small.py
- python/sglang/srt/models/qwen.py
- python/sglang/srt/models/qwen2.py
- python/sglang/srt/models/qwen2_moe.py
- python/sglang/srt/models/stablelm.py
- python/sglang/srt/models/xverse.py
- python/sglang/srt/models/xverse_moe.py
- python/sglang/test/test_utils.py
head_ref_oid: f1ab62778d2938f79dedf0f0a49ff5ac1ce662e9
artifact_dir: evidence/pull-bundles/sglang/gh-4686
---
# Fix loading KV quantization scale; Enable modelopt kv cache

- Repository: `sgl-project/sglang`
- PR: [4686](https://github.com/sgl-project/sglang/pull/4686)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `python/sglang/srt/configs/model_config.py`
- `python/sglang/srt/layers/attention/flashattention_backend.py`
- `python/sglang/srt/layers/attention/flashinfer_backend.py`
- `python/sglang/srt/layers/quantization/kv_cache.py`
- `python/sglang/srt/layers/quantization/modelopt_quant.py`
- `python/sglang/srt/layers/radix_attention.py`
- `python/sglang/srt/models/baichuan.py`
- `python/sglang/srt/models/chatglm.py`
- `python/sglang/srt/models/commandr.py`
- `python/sglang/srt/models/dbrx.py`
- `python/sglang/srt/models/deepseek.py`
- `python/sglang/srt/models/deepseek_v2.py`
- `python/sglang/srt/models/exaone.py`
- `python/sglang/srt/models/gemma.py`
- `python/sglang/srt/models/gemma2.py`
- `python/sglang/srt/models/gemma3_causal.py`
- `python/sglang/srt/models/gpt2.py`
- `python/sglang/srt/models/gpt_bigcode.py`
- `python/sglang/srt/models/granite.py`
- `python/sglang/srt/models/grok.py`
- `python/sglang/srt/models/internlm2.py`
- `python/sglang/srt/models/llama.py`
- `python/sglang/srt/models/minicpm.py`
- `python/sglang/srt/models/minicpm3.py`
- `python/sglang/srt/models/mixtral.py`
- `python/sglang/srt/models/mixtral_quant.py`
- `python/sglang/srt/models/mllama.py`
- `python/sglang/srt/models/olmo.py`
- `python/sglang/srt/models/olmo2.py`
- `python/sglang/srt/models/olmoe.py`
- `python/sglang/srt/models/phi3_small.py`
- `python/sglang/srt/models/qwen.py`
- `python/sglang/srt/models/qwen2.py`
- `python/sglang/srt/models/qwen2_moe.py`
- `python/sglang/srt/models/stablelm.py`
- `python/sglang/srt/models/xverse.py`
- `python/sglang/srt/models/xverse_moe.py`
- `python/sglang/test/test_utils.py`

## Evidence Bundle

`evidence/pull-bundles/sglang/gh-4686`
