Commits · 0e82fd3df4b2f36e3352b72ec2a441d9efed3a5f · change / sglang · GitLab

04 Nov, 2025 23 commits
- [router][grpc] Fix model validation, tool call check, streaming logic and misc... · 0e82fd3d
  Chang Su authored Nov 04, 2025
```
[router][grpc] Fix model validation, tool call check, streaming logic and misc in responses  (#12616)
```
  0e82fd3d
- Add sanity checks when a test file is not added to CI (reland) (#12594) · b7d70411
  fzyzcjy authored Nov 04, 2025
  
  b7d70411
- Ensure GPU work is finished when release memory occupation call is finished (#12592) · ff0b64e1
  fzyzcjy authored Nov 04, 2025
  
  ff0b64e1
- Support aggregating engine metrics in sgl-router (#11456) · d84790db
  fzyzcjy authored Nov 04, 2025
  
  d84790db
- [sepc-v2] Fix imcompatibility with constrained decoding (#12615) · 0678beaa
  Liangsheng Yin authored Nov 04, 2025
  
  0678beaa
- chore: bump mooncake version to 0.3.7.post2 (#12599) · c2d4716d
  Shangming Cai authored Nov 04, 2025
```
Signed-off-by: Shangming Cai <csmthu@gmail.com>
```
  c2d4716d
- [Deterministic] Optimize bmm_batch_invariant op (#12522) · c14cc47e
  Minglei Zhu authored Nov 04, 2025
  
  c14cc47e
- Add --speculative-moe-runner-backend server arg (#10183) · dbcf85b7
  Trevor Morris authored Nov 04, 2025
  
  dbcf85b7
- [router][grpc] Restructure modules and code clean up (#12598) · 83804bc6
  Chang Su authored Nov 03, 2025
  
  83804bc6
- feat: limit peak memory usage when computing logprobs (#6318) · d5fa019c
  Zhao Chen authored Nov 04, 2025
```
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
```
  d5fa019c
- Restore torch defaults between sgl-kernel tests (#11131) · fef3a6b6
  Ben Barsdell authored Nov 04, 2025
  
  fef3a6b6
- Enable memory saver for hybrid model (#11974) · 173e0f70
  Junrong Lin authored Nov 04, 2025
  
  173e0f70
- Improve the metrics for PD (#12580) · f600866a
  Lianmin Zheng authored Nov 03, 2025
```
Co-authored-by: Kan Wu <wukanustc@gmail.com>
Co-authored-by: cctry <shiyang@x.ai>
```
  f600866a
- fix: respect `--ignore-eos` in PD case for benchmarking (#12597) · 93be7e86
  ishandhanani authored Nov 03, 2025
  
  93be7e86
- Tiny fix ExpertDistributionReq error (#11760) · 60b0754c
  fzyzcjy authored Nov 04, 2025
  
  60b0754c
- test: support return logprobs in bench_offline_throughput test (#12462) · 0b24af4d
  Zhao Chen authored Nov 04, 2025
```
Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  0b24af4d
- [Qwen3 VL] Add LoRA support for Qwen 3 VL (#12165) · a209fb05
  Jonah Bernard authored Nov 03, 2025
  
  a209fb05
- [GDN/SWA] mamba and swa radix cache edge case fix (#12111) · 48d6bea1
  Hanming Lu authored Nov 03, 2025
```
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  48d6bea1
- [Doc] fix miss index for production request trace (#12547) · 1689c0e3
  Teng Ma authored Nov 04, 2025
  
  1689c0e3
- Super tiny add UT for copy_to_gpu_no_ce (#12270) · 193fbb0b
  fzyzcjy authored Nov 04, 2025
  
  193fbb0b
- Enable mixed type LayerNorm kernel for NSA indexer (#12044) · e607850f
  akhilg-nv authored Nov 03, 2025
  
  e607850f
- [chore] Fix update_kernel_whl_index script for multiple cuda version (#12519) · 15efbcb4
  Baizhou Zhang authored Nov 03, 2025
  
  15efbcb4
- Remove the dependency of nccl.h in symmetric memory (#12571) · 243c064d
  Lianmin Zheng authored Nov 03, 2025
  
  243c064d
03 Nov, 2025 17 commits
- [router][grpc] Consolidate error messages build in error.rs (#12301) · 0b41a293
  Chang Su authored Nov 03, 2025
  
  0b41a293
- update usage of `trtllm_fp8_per_tensor_scale_moe` (#12569) · d31d48b3
  b8zhong authored Nov 03, 2025
  
  d31d48b3
- Super tiny dump server info such as args in bench for post analysis (#12550) · 88342607
  fzyzcjy authored Nov 04, 2025
  
  88342607
- Super tiny allow profile activities in bench_serving (#12549) · fd7a72d6
  fzyzcjy authored Nov 04, 2025
  
  fd7a72d6
- tiny optimize for bench serving (#12553) · 21a8fa16
  Yi Zhang authored Nov 04, 2025
  
  21a8fa16
- Reduce the overhead of nccl symmetric memory (#12524) · 7a21d8b2
  Lianmin Zheng authored Nov 03, 2025
```
Co-authored-by: Nicolas Castet <ncastet@nvidia.com>
```
  7a21d8b2
- [ROCm] Update Mooncake to v0.3.7.post1 and add -DUSE_HIP=ON to rocm.Dockerfile (#12560) · d36639ee
  R0CKSTAR authored Nov 04, 2025
```
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
```
  d36639ee
- [Test] Add parameters to SRTRunner (#12227) · 6ef23b98
  Jonah Bernard authored Nov 03, 2025
  
  6ef23b98
- Fix error when calling quantization (#12548) · 385599cb
  fzyzcjy authored Nov 04, 2025
  
  385599cb
- fix: fix the bug which leads qwen2_5_vl to crash with mixed_chunk (#11330) · 952fbe47
  Yueyang Pan authored Nov 03, 2025
```
Signed-off-by: PanJason <pyyjason@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
```
  952fbe47
- [hot-fix] Fix broken CI (#12564) · edb25693
  Liangsheng Yin authored Nov 04, 2025
  
  edb25693
- [spec v2] Fix output repetition by speculative sampling error (#12561) · 3529c061
  Liangsheng Yin authored Nov 03, 2025
  
  3529c061
- Conditionally recapture cuda graph after model weight update from disk (#12060) · ffb32a85
  harrisonlimh authored Nov 03, 2025
  
  ffb32a85
- fix: Fix KTransformers hybrid inference with int8 quantization and format (#12536) · 14d80648
  Atream authored Nov 03, 2025
  
  14d80648
- chore: upgrade mooncake 0.3.7.post1 (#12541) · ab8b83f7
  Shangming Cai authored Nov 03, 2025
```
Signed-off-by: Shangming Cai <csmthu@gmail.com>
```
  ab8b83f7
- fix: move dummy format loader check before quantization checks (#12532) · de0b10cf
  yinghui authored Nov 02, 2025
  
  de0b10cf
- [hotfix] Remove flashinfer-jit-cache from pyproject (#12530) · 6e29446e
  Baizhou Zhang authored Nov 02, 2025
  
  6e29446e