Commits · 83804bc62647fbccc5befb568c55b1d85b3393fd · change / sglang · GitLab

04 Nov, 2025 15 commits
- [router][grpc] Restructure modules and code clean up (#12598) · 83804bc6
  Chang Su authored Nov 03, 2025
  
  83804bc6
- feat: limit peak memory usage when computing logprobs (#6318) · d5fa019c
  Zhao Chen authored Nov 04, 2025
```
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
```
  d5fa019c
- Restore torch defaults between sgl-kernel tests (#11131) · fef3a6b6
  Ben Barsdell authored Nov 04, 2025
  
  fef3a6b6
- Enable memory saver for hybrid model (#11974) · 173e0f70
  Junrong Lin authored Nov 04, 2025
  
  173e0f70
- Improve the metrics for PD (#12580) · f600866a
  Lianmin Zheng authored Nov 03, 2025
```
Co-authored-by: Kan Wu <wukanustc@gmail.com>
Co-authored-by: cctry <shiyang@x.ai>
```
  f600866a
- fix: respect `--ignore-eos` in PD case for benchmarking (#12597) · 93be7e86
  ishandhanani authored Nov 03, 2025
  
  93be7e86
- Tiny fix ExpertDistributionReq error (#11760) · 60b0754c
  fzyzcjy authored Nov 04, 2025
  
  60b0754c
- test: support return logprobs in bench_offline_throughput test (#12462) · 0b24af4d
  Zhao Chen authored Nov 04, 2025
```
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  0b24af4d
- [Qwen3 VL] Add LoRA support for Qwen 3 VL (#12165) · a209fb05
  Jonah Bernard authored Nov 03, 2025
  
  a209fb05
- [GDN/SWA] mamba and swa radix cache edge case fix (#12111) · 48d6bea1
  Hanming Lu authored Nov 03, 2025
```
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  48d6bea1
- [Doc] fix miss index for production request trace (#12547) · 1689c0e3
  Teng Ma authored Nov 04, 2025
  
  1689c0e3
- Super tiny add UT for copy_to_gpu_no_ce (#12270) · 193fbb0b
  fzyzcjy authored Nov 04, 2025
  
  193fbb0b
- Enable mixed type LayerNorm kernel for NSA indexer (#12044) · e607850f
  akhilg-nv authored Nov 03, 2025
  
  e607850f
- [chore] Fix update_kernel_whl_index script for multiple cuda version (#12519) · 15efbcb4
  Baizhou Zhang authored Nov 03, 2025
  
  15efbcb4
- Remove the dependency of nccl.h in symmetric memory (#12571) · 243c064d
  Lianmin Zheng authored Nov 03, 2025
  
  243c064d
03 Nov, 2025 21 commits
- [router][grpc] Consolidate error messages build in error.rs (#12301) · 0b41a293
  Chang Su authored Nov 03, 2025
  
  0b41a293
- update usage of `trtllm_fp8_per_tensor_scale_moe` (#12569) · d31d48b3
  b8zhong authored Nov 03, 2025
  
  d31d48b3
- Super tiny dump server info such as args in bench for post analysis (#12550) · 88342607
  fzyzcjy authored Nov 04, 2025
  
  88342607
- Super tiny allow profile activities in bench_serving (#12549) · fd7a72d6
  fzyzcjy authored Nov 04, 2025
  
  fd7a72d6
- tiny optimize for bench serving (#12553) · 21a8fa16
  Yi Zhang authored Nov 04, 2025
  
  21a8fa16
- Reduce the overhead of nccl symmetric memory (#12524) · 7a21d8b2
  Lianmin Zheng authored Nov 03, 2025
```
Co-authored-by: Nicolas Castet <ncastet@nvidia.com>
```
  7a21d8b2
- [ROCm] Update Mooncake to v0.3.7.post1 and add -DUSE_HIP=ON to rocm.Dockerfile (#12560) · d36639ee
  R0CKSTAR authored Nov 04, 2025
```
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
```
  d36639ee
- [Test] Add parameters to SRTRunner (#12227) · 6ef23b98
  Jonah Bernard authored Nov 03, 2025
  
  6ef23b98
- Fix error when calling quantization (#12548) · 385599cb
  fzyzcjy authored Nov 04, 2025
  
  385599cb
- fix: fix the bug which leads qwen2_5_vl to crash with mixed_chunk (#11330) · 952fbe47
  Yueyang Pan authored Nov 03, 2025
```
Signed-off-by: PanJason <pyyjason@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
```
  952fbe47
- [hot-fix] Fix broken CI (#12564) · edb25693
  Liangsheng Yin authored Nov 04, 2025
  
  edb25693
- [spec v2] Fix output repetition by speculative sampling error (#12561) · 3529c061
  Liangsheng Yin authored Nov 03, 2025
  
  3529c061
- Conditionally recapture cuda graph after model weight update from disk (#12060) · ffb32a85
  harrisonlimh authored Nov 03, 2025
  
  ffb32a85
- fix: Fix KTransformers hybrid inference with int8 quantization and format (#12536) · 14d80648
  Atream authored Nov 03, 2025
  
  14d80648
- chore: upgrade mooncake 0.3.7.post1 (#12541) · ab8b83f7
  Shangming Cai authored Nov 03, 2025
```
Signed-off-by: Shangming Cai <csmthu@gmail.com>
```
  ab8b83f7
- fix: move dummy format loader check before quantization checks (#12532) · de0b10cf
  yinghui authored Nov 02, 2025
  
  de0b10cf
- [hotfix] Remove flashinfer-jit-cache from pyproject (#12530) · 6e29446e
  Baizhou Zhang authored Nov 02, 2025
  
  6e29446e
- chore: upgrade flashinfer 0.5.0 (#12523) · 0c3543d7
  Yineng Zhang authored Nov 02, 2025
```
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  0c3543d7
- Update setup_github_runner.md · 6a3b9fd0
  Kangyan-Zhou authored Nov 02, 2025
  
  6a3b9fd0
- [Bug] Fix Intern-S1 model accuracy and support /generate interface with input_ids (#12367) · 65f1d065
  Haian Huang(深度眸) authored Nov 03, 2025
  
  65f1d065
- [Refact] Remove hardcoded KV cache dimension in MLATokenToKVPool (#12502) · 9434a0e5
  Johnsonms authored Nov 02, 2025
  
  9434a0e5
02 Nov, 2025 4 commits
- move all get_stream in sgl_kernel to c++ to reduce the launch overhead (#12521) · 20315697
  Lianmin Zheng authored Nov 02, 2025
  
  20315697
- Super tiny fix naming in bench serving scripts (#12515) · c9db7911
  fzyzcjy authored Nov 03, 2025
  
  c9db7911
- [Fix] `concat_mla_absorb_q_kernel` fails for long inputs (#12453) · 15ed27d7
  bingps authored Nov 03, 2025
  
  15ed27d7
- [ServerArgs] allow --mamba-ssm-dtype extend (#12481) · 66fb9b13
  Hanming Lu authored Nov 02, 2025
  
  66fb9b13