Commits · 21a8fa16ea5dec9d586f170caa0620a2c0cfbdbe · change / sglang · GitLab

03 Nov, 2025 17 commits
- tiny optimize for bench serving (#12553) · 21a8fa16
  Yi Zhang authored Nov 04, 2025
  
  21a8fa16
- Reduce the overhead of nccl symmetric memory (#12524) · 7a21d8b2
  Lianmin Zheng authored Nov 03, 2025
```
Co-authored-by: Nicolas Castet <ncastet@nvidia.com>
```
  7a21d8b2
- [ROCm] Update Mooncake to v0.3.7.post1 and add -DUSE_HIP=ON to rocm.Dockerfile (#12560) · d36639ee
  R0CKSTAR authored Nov 04, 2025
```
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
```
  d36639ee
- [Test] Add parameters to SRTRunner (#12227) · 6ef23b98
  Jonah Bernard authored Nov 03, 2025
  
  6ef23b98
- Fix error when calling quantization (#12548) · 385599cb
  fzyzcjy authored Nov 04, 2025
  
  385599cb
- fix: fix the bug which leads qwen2_5_vl to crash with mixed_chunk (#11330) · 952fbe47
  Yueyang Pan authored Nov 03, 2025
```
Signed-off-by: PanJason <pyyjason@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
```
  952fbe47
- [hot-fix] Fix broken CI (#12564) · edb25693
  Liangsheng Yin authored Nov 04, 2025
  
  edb25693
- [spec v2] Fix output repetition by speculative sampling error (#12561) · 3529c061
  Liangsheng Yin authored Nov 03, 2025
  
  3529c061
- Conditionally recapture cuda graph after model weight update from disk (#12060) · ffb32a85
  harrisonlimh authored Nov 03, 2025
  
  ffb32a85
- fix: Fix KTransformers hybrid inference with int8 quantization and format (#12536) · 14d80648
  Atream authored Nov 03, 2025
  
  14d80648
- chore: upgrade mooncake 0.3.7.post1 (#12541) · ab8b83f7
  Shangming Cai authored Nov 03, 2025
```
Signed-off-by: Shangming Cai <csmthu@gmail.com>
```
  ab8b83f7
- fix: move dummy format loader check before quantization checks (#12532) · de0b10cf
  yinghui authored Nov 02, 2025
  
  de0b10cf
- [hotfix] Remove flashinfer-jit-cache from pyproject (#12530) · 6e29446e
  Baizhou Zhang authored Nov 02, 2025
  
  6e29446e
- chore: upgrade flashinfer 0.5.0 (#12523) · 0c3543d7
  Yineng Zhang authored Nov 02, 2025
```
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  0c3543d7
- Update setup_github_runner.md · 6a3b9fd0
  Kangyan-Zhou authored Nov 02, 2025
  
  6a3b9fd0
- [Bug] Fix Intern-S1 model accuracy and support /generate interface with input_ids (#12367) · 65f1d065
  Haian Huang(深度眸) authored Nov 03, 2025
  
  65f1d065
- [Refact] Remove hardcoded KV cache dimension in MLATokenToKVPool (#12502) · 9434a0e5
  Johnsonms authored Nov 02, 2025
  
  9434a0e5
02 Nov, 2025 16 commits
- move all get_stream in sgl_kernel to c++ to reduce the launch overhead (#12521) · 20315697
  Lianmin Zheng authored Nov 02, 2025
  
  20315697
- Super tiny fix naming in bench serving scripts (#12515) · c9db7911
  fzyzcjy authored Nov 03, 2025
  
  c9db7911
- [Fix] `concat_mla_absorb_q_kernel` fails for long inputs (#12453) · 15ed27d7
  bingps authored Nov 03, 2025
  
  15ed27d7
- [ServerArgs] allow --mamba-ssm-dtype extend (#12481) · 66fb9b13
  Hanming Lu authored Nov 02, 2025
  
  66fb9b13
- Add prefix for torch symm mem (#12506) · 819fc591
  Yuan Luo authored Nov 03, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
```
  819fc591
- [FEAT] Shared mem pool based cuda ipc for multi-modal data transport (#11917) · 7efd8b3d
  kousakawang authored Nov 02, 2025
```
Co-authored-by: kousakawang <wanghanpei@bytedance.com>
Co-authored-by: Yuan Luo <4908075+yuan-luo@users.noreply.github.com>
```
  7efd8b3d
- chore: bump sgl-kernel version to 0.3.16.post5 (#12511) · a920b9da
  sglang-bot authored Nov 02, 2025
  
  a920b9da
- feat: Add FP4 (E2M1) KV Cache Support with Quantization Utilities for MLA (#10078) · 76196b3c
  Ho-Ren (Jack) Chuang authored Nov 01, 2025
```
Signed-off-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
Co-authored-by: Yichen Wang <yichen.wang@bytedance.com>
```
  76196b3c
- Migrate weak_ref_tensor to sgl-kernel (#12505) · 95191ebd
  Xiaoyu Zhang authored Nov 02, 2025
  
  95191ebd
- [CI] Move some Lora/Deterministic CI tests to nightly (#12507) · 9a512cf9
  Baizhou Zhang authored Nov 01, 2025
  
  9a512cf9
- [Feature] Qwen3-Next & FLA: Support MTP topk>1; Up to 6% faster (#11133) · 3451fc32
  Binyao Jiang authored Nov 01, 2025
```
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
```
  3451fc32
- [Ascend] Add Ascend NPU support for sglang.check_env & rework proposal (#11052) · c550ab91
  Zhihao Lyu authored Nov 02, 2025
```
Co-authored-by: ronnie_zheng <zl19940307@163.com>
```
  c550ab91
- [CI][NPU] remove pypi mirror site that hangs ci dependency installation (#12499) · 086f0b79
  Even Zhou authored Nov 02, 2025
  
  086f0b79
- Update Mooncake EP's a2a interface (#12391) · 0afd6832
  Xun Sun authored Nov 02, 2025
  
  0afd6832
- [Bug] test_flashattn_mla_backend errors in Hopper #12487 (#12488) · 6f858930
  Johnsonms authored Nov 01, 2025
  
  6f858930
- [Deterministic] add deepseek v3 deterministic inference CI test (#12412) · 229256c5
  Minglei Zhu authored Nov 01, 2025
  
  229256c5
01 Nov, 2025 7 commits
- [HICache / PD]: Support offloading incremental KV cache in decode side. (#11966) · 6b634493
  hzh0425 authored Nov 02, 2025
  
  6b634493
- Reduce docker image size. mount cache when use pip/cargo build (#12238) · 756ad9ce
  ybyang authored Nov 02, 2025
```
Signed-off-by: ybyang <ybyang7@iflytek.com>
```
  756ad9ce
- [feat] Add SGLANG_TOOL_STRICT_LEVEL for tool-call behavior control (#12423) · d2a8f71c
  Xinyuan Tong authored Nov 01, 2025
```
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
```
  d2a8f71c
- [Hotfix] Remove extra comment in sgl-kernel README (#12500) · 2b7bf11b
  Baizhou Zhang authored Nov 01, 2025
  
  2b7bf11b
- [CI] Build aarch64 kernels for sgl-kernel test (#12480) · 566ade03
  Baizhou Zhang authored Nov 01, 2025
  
  566ade03
- Filter tokenizer warning for kimi models (#12485) · 69193f71
  Ke Bao authored Nov 01, 2025
  
  69193f71
- perf: trtllm mla performance minor improvements (#12435) · d5b6e50f
  yinghui authored Oct 31, 2025
  
  d5b6e50f