Commits · 83871aa12df1fcd478ab341c76b1cf043facab1d · change / sglang · GitLab

23 Aug, 2025 11 commits
- feat(hicache): Supports 3fs-hicache compatibility with dp-attention (#9372) · 83871aa1
  hzh0425 authored Aug 23, 2025
  
  83871aa1
- Partially unify triton per token group quant kernels (#9485) · b1b3f0b3
  fzyzcjy authored Aug 23, 2025
  
  b1b3f0b3
- Tiny make device_loading_context more static (#9478) · 34e5e11f
  fzyzcjy authored Aug 23, 2025
  
  34e5e11f
- Overlapped weight offload (#8034) · 2600fc0d
  fzyzcjy authored Aug 23, 2025
  
  2600fc0d
- [fix] Fix mxfp4 triton MoE tp bug (#9473) · ccd3fb94
  hlu1 authored Aug 23, 2025
```
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
```
  ccd3fb94
- tool-call(dsv3): Improve deepseek-v3 chat template and tool_choice = `required` (#9525) · c9dd70fb
  Chang Su authored Aug 23, 2025
  
  c9dd70fb
- fix: blackwell dsv3 fp8 issue temporary solution (#9530) · 6b2b8bf0
  Yineng Zhang authored Aug 23, 2025
  
  6b2b8bf0
- [benchmark] Add benchmark scripts for ceval and boolq (#8946) · 4edbe0d5
  yuxingcyx authored Aug 23, 2025
```
Co-authored-by: chenyuxing <2818499974@qq.com>
Co-authored-by: hanqing <huang010706@126.com>
Co-authored-by: Muggle <62579327+trawolf@users.noreply.github.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
```
  4edbe0d5
- Add enable_flashinfer_mxfp4_bf16_moe for higher precision and slower moe backend (#9004) · 0374304a
  fzyzcjy authored Aug 23, 2025
  
  0374304a
- Support GC Freezing to improve latency & throughput (#9241) · 127d4b0d
  Chanh Nguyen authored Aug 22, 2025
```
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
```
  127d4b0d
- Add support for extensions of interface and pre-registrations to NIXL HiCache (#9211) · 7e880286
  Moein Khazraee authored Aug 22, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  7e880286
22 Aug, 2025 29 commits
- [router] ignore client error when record failure in pd_router (#9503) · 446c8e4c
  Bruce-x-1997 authored Aug 23, 2025
```
Co-authored-by: bruce.xu <bruce.xu@gmicloud.ai>
```
  446c8e4c
- [router] Move all protocols to spec.rs file (#9519) · 5ef545e6
  Keyang Ru authored Aug 22, 2025
  
  5ef545e6
- Add Qwen3-30B-A3B-Thinking-2507 support on AMD GPUs. (#9456) · c4500233
  sogalin authored Aug 22, 2025
  
  c4500233
- [AMD] Fix Llama 4 FP8 accuracy issues on MI300X (#7699) · f445a1d9
  Hubert Lu authored Aug 22, 2025
  
  f445a1d9
- [NVIDA] [1/N] Nvfp4 Masked Gemm: Add quant op for the flashinfer grouped gemm (#9200) · e5638573
  Kaixi Hou authored Aug 22, 2025
  
  e5638573
- [router] add json tool parser (#9516) · f556ac8b
  Simo Lin authored Aug 22, 2025
  
  f556ac8b
- [MTP] Force greedy sampling on AMD (#9127) · 110a6598
  datdo-msft authored Aug 22, 2025
  
  110a6598
- [router] tokenizer arch doc (#9513) · 49f9d025
  Simo Lin authored Aug 22, 2025
  
  49f9d025
- Use Tensor Core Decode when gqa group size >= 4 (#8624) · 0f587e80
  Wenxuan Tan authored Aug 22, 2025
```
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  0f587e80
- [HiCacheStorage] backup optimization for MLA model (#8865) · 6078d5fc
  huangtingwei authored Aug 22, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  6078d5fc
- 3fs zerocopy (#9109) · 70cf4abc
  pansicheng authored Aug 22, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  70cf4abc
- [bugfix] Make --enable-hierarchical-cache and --disable-radix-cache mutually exclusive (#9452) · cebf4599
  Xuchun Shang authored Aug 22, 2025
```
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
```
  cebf4599
- Disable torch.compile for get_last_loc_large_page_size_large_top_k (#9507) · 9c0c1e30
  Qiaolin Yu authored Aug 22, 2025
```
Co-authored-by: ispobock <ispobaoke@gmail.com>
```
  9c0c1e30
- minor: determine mm attn backend based on platforms (#9303) · a1f011d0
  Mick authored Aug 22, 2025
  
  a1f011d0
- Support speculative decoding in the trtllm_mha attention backend (#9331) · 9ec314c6
  Qiaolin Yu authored Aug 21, 2025
```
Co-authored-by: ispobock <ispobaoke@gmail.com>
```
  9ec314c6
- [Docs] Add doc and quick demo for gpt-oss responses api & buildin tools (#9497) · fedfe91c
  Xinyuan Tong authored Aug 22, 2025
```
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
```
  fedfe91c
- Update docker file for supporting PD-Disaggregation on MI300x (#9494) · 988accbc
  kk authored Aug 22, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Colin Wang <kangwang@amd.com>
```
  988accbc
- chore: bump sgl-kernel v0.3.6.post2 (#9475) · b6b2287e
  Yineng Zhang authored Aug 21, 2025
  
  b6b2287e
- Add trtllm_mla and cutlass_mla for ragged fmha for chunked prefill (#9480) · 243e745d
  Elfie Guo authored Aug 21, 2025
  
  243e745d
- torch.compile() mrope (#9487) · 61a0e600
  timmy-feng authored Aug 22, 2025
  
  61a0e600
- [router] fix router load guard tracking for streaming (#9491) · 0f8cee8c
  Simo Lin authored Aug 21, 2025
  
  0f8cee8c
- [router] add tool parser base structure and partial json parser (#9482) · 816c4c85
  Chang Su authored Aug 21, 2025
  
  816c4c85
- [Docs]Update reasoning parser doc & fix outdated link (#9492) · 13ec8d42
  Xinyuan Tong authored Aug 22, 2025
```
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
```
  13ec8d42
- [docs]: fix reasoning context in docs (#9483) · 05bd7897
  Chayenne authored Aug 21, 2025
  
  05bd7897
- [code clean] add H20 cutlass groupGemm default config (#9333) · 5fd311d3
  kousakawang authored Aug 22, 2025
```
Co-authored-by: wanghanpei <wanghanpei@bytedance.com>
```
  5fd311d3
- [router] remove all tokenizer metrics for performance (#9474) · 53e2cd46
  Chang Su authored Aug 21, 2025
  
  53e2cd46
- Support MHA with chunked prefix cache for flashinfer/flashmla backend, support... · 9708d353
  Yongfei Xu authored Aug 22, 2025
```
Support MHA with chunked prefix cache for flashinfer/flashmla backend, support page size > 1 for MHA chunked prefix (#8616)
Co-authored-by: xuyongfei.xyf <xuyongfei.xyf@antgroup.com>
```
  9708d353
- [AMD] Remove the deprecated C10_WARP_SIZE (#9356) · 704ced1b
  Hubert Lu authored Aug 21, 2025
  
  704ced1b
- Add Support for Page Size greater than 1 for Flashinfer MLA Backend (#8593) · 3cc3d9b9
  Pavani Majety authored Aug 21, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  3cc3d9b9