Commits · 0e0eef00ce465256d3839018371bd98b9a82df3e · change / sglang · GitLab

03 Aug, 2025 18 commits
- [DP] fix the compatibility issue between DP attention and `--attention-backend triton` (#8723) · 0e0eef00
  Cheng Wan authored Aug 03, 2025
  
  0e0eef00
- [CUDA Graph] save cuda graph memory by using next_token_logits_buffer (#8579) · cb099d20
  Cheng Wan authored Aug 03, 2025
  
  cb099d20
- Save cuda graph memory for fa3 (#8567) · 7a913301
  Cheng Wan authored Aug 03, 2025
  
  7a913301
- [Improvements] Merge health check route (#8444) · 6f9baf10
  ybyang authored Aug 03, 2025
```
Signed-off-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Kan Wu <wukanustc@gmail.com>
```
  6f9baf10
- feat: Add new moe triton for NVIDIA RTX 6000 Ada (#8547) · a31b7a70
  Jasper James authored Aug 03, 2025
  
  a31b7a70
- [fix] Fix divide by zero error for llama4. (#8683) · 7ed8e51b
  Varun Vinayak Shenoy authored Aug 03, 2025
  
  7ed8e51b
- Do layernorm before allgather for DP attention (#8631) · 32f28154
  Trevor Morris authored Aug 03, 2025
  
  32f28154
- [feat] support minimum token load balance in dp attention (#7379) · f7b2853f
  Guanhua Wang authored Aug 03, 2025
  
  f7b2853f
- HiCache storage, style change and bug fix (#8719) · b0add2da
  Zhiqiang Xie authored Aug 03, 2025
  
  b0add2da
- Reduce memory accumulation in long-running server (#8306) · 0305c505
  Wenxuan Tan authored Aug 03, 2025
```
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
```
  0305c505
- Support limiting max loaded loras in CPU. (#8650) · 8675bdf2
  Lifu Huang authored Aug 03, 2025
  
  8675bdf2
- [hotfix] fix mixtral with tensor-level compressed-tensor quantization (#8721) · a437aa99
  Cheng Wan authored Aug 02, 2025
  
  a437aa99
- Tiny fix CI pytest error (#8524) · 0e612dbf
  fzyzcjy authored Aug 03, 2025
  
  0e612dbf
- Fix fused MoE when `routed_scaling_factor is None` (#8709) · 9f47d686
  Liangsheng Yin authored Aug 03, 2025
  
  9f47d686
- [Feature] Radix Tree in C++ (#7369) · e273aa6d
  DarkSharpness authored Aug 02, 2025
  
  e273aa6d
- Fix triton moe error caused by TopK refactor (#8705) · 8ada1ab6
  fzyzcjy authored Aug 03, 2025
  
  8ada1ab6
- [FIX] Fix the nightly CI by disabling swa mem pool for gemma2 (#8693) · e314b084
  Lianmin Zheng authored Aug 02, 2025
  
  e314b084
- Remove assertions about per group quant fp8 (#8717) · 403566bc
  fzyzcjy authored Aug 03, 2025
  
  403566bc
02 Aug, 2025 4 commits
- Add tensor.detach() back to update weight util (#8691) · 4ca43b06
  Stefan He authored Aug 02, 2025
  
  4ca43b06
- model: adapt mllama4 to VisionAttention (#8512) · ea93079b
  Wenchen Lo authored Aug 02, 2025
```
Co-authored-by: root <mickjagger19@icloud.com>
```
  ea93079b
- Fix: resolve prefill of retracted request out-of-memory issue when ignore_eos is enabled (#7434) · 4bec99ec
  Yusong Gao authored Aug 02, 2025
  
  4bec99ec
- [bugfix] Apply routed scaling factor to cutlass_fused_experts_fp8 (#8688) · 89caf7a3
  Trevor Morris authored Aug 01, 2025
  
  89caf7a3
01 Aug, 2025 18 commits
- Add support for NCCL symmetric memory for TP allreduces (#8238) · 82e6c3a6
  Nicolas Castet authored Aug 01, 2025
  
  82e6c3a6
- [bugfix] Add 'disaggregation_mode' parameter to warmup function when compile... · b89d37cb
  Baron Liu authored Aug 02, 2025
```
[bugfix] Add 'disaggregation_mode' parameter to warmup function when compile deep_gemm manually (#8618)
```
  b89d37cb
- upgrade xgrammar 0.1.22 (#8522) · 5deab128
  Swipe4057 authored Aug 02, 2025
  
  5deab128
- bugfix(hicache): Fix 'MooncakeStore' not defined error. (#8668) · d1c4d51c
  hzh0425 authored Aug 02, 2025
  
  d1c4d51c
- Fix deepgemm masked grouped gemm jit compile (#8679) · e2521926
  Ke Bao authored Aug 02, 2025
  
  e2521926
- [bugfix] Fix page size for create_flashmla_kv_indices_triton() for cutlass mla (#8685) · 6a7528e6
  Trevor Morris authored Aug 01, 2025
  
  6a7528e6
- Disable tp for shared experts under expert parallelism for GLM4.5 model (#8647) (#8647) · 2ae95d17
  Minglei Zhu authored Aug 01, 2025
```
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  2ae95d17
- [fix] fix pd disagg error of vlms (#8094) · 2d401bd9
  萝卜菜 authored Aug 02, 2025
  
  2d401bd9
- [5/N] MoE Refactor: Update MoE parallelism arguments (#8658) · 6c88f6c8
  Cheng Wan authored Aug 01, 2025
  
  6c88f6c8
- Bug: apply final_hidden_states*=self.routed_scaling_factor at MoE lay… (#8511) · c8d3a402
  Binyao Jiang authored Aug 01, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  c8d3a402
- Fix chat template handling for OpenAI serving (#8635) · 7e831efe
  Xinyuan Tong authored Jul 31, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
```
  7e831efe
- Add hf3fs_utils.cpp to package-data (#8653) · 20b5563e
  pansicheng authored Aug 01, 2025
  
  20b5563e
- chore: bump v0.4.10.post1 (#8652) · 33f0de33
  Ke Bao authored Aug 01, 2025
  
  33f0de33
- Update batch size limitation of dsv3_router_gemm kernel to 16 (#8051) · e7e5a305
  Baizhou Zhang authored Jul 31, 2025
  
  e7e5a305
- Interface change for kvcache io to support page first layout (#8318) · dd7ca006
  Zhiqiang Xie authored Jul 31, 2025
  
  dd7ca006
- HiCache, fixing hash value indexing (#8636) · 9305ea6c
  Zhiqiang Xie authored Jul 31, 2025
  
  9305ea6c
- [NVIDIA] Enable Flashinfer MoE blockscale fp8 backend for TP MoE (#8450) · aa4c66b5
  Kaixi Hou authored Jul 31, 2025
```
Co-authored-by: kushanam <42385577+kushanam@users.noreply.github.com>
```
  aa4c66b5
- [Bugfix] fix w8a8_int8 load issue (#8308) · 99795d61
  Even Zhou authored Aug 01, 2025
```
Co-authored-by: ronnie_zheng <zl19940307@163.com>
```
  99795d61