Commits · 0f587e80d3538c2816ae243bfb8bda8e1b08cab9 · change / sglang · GitLab

22 Aug, 2025 25 commits
- Use Tensor Core Decode when gqa group size >= 4 (#8624) · 0f587e80
  Wenxuan Tan authored Aug 22, 2025
```
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  0f587e80
- [HiCacheStorage] backup optimization for MLA model (#8865) · 6078d5fc
  huangtingwei authored Aug 22, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  6078d5fc
- 3fs zerocopy (#9109) · 70cf4abc
  pansicheng authored Aug 22, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  70cf4abc
- [bugfix] Make --enable-hierarchical-cache and --disable-radix-cache mutually exclusive (#9452) · cebf4599
  Xuchun Shang authored Aug 22, 2025
```
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
```
  cebf4599
- Disable torch.compile for get_last_loc_large_page_size_large_top_k (#9507) · 9c0c1e30
  Qiaolin Yu authored Aug 22, 2025
```
Co-authored-by: ispobock <ispobaoke@gmail.com>
```
  9c0c1e30
- minor: determine mm attn backend based on platforms (#9303) · a1f011d0
  Mick authored Aug 22, 2025
  
  a1f011d0
- Support speculative decoding in the trtllm_mha attention backend (#9331) · 9ec314c6
  Qiaolin Yu authored Aug 21, 2025
```
Co-authored-by: ispobock <ispobaoke@gmail.com>
```
  9ec314c6
- [Docs] Add doc and quick demo for gpt-oss responses api & buildin tools (#9497) · fedfe91c
  Xinyuan Tong authored Aug 22, 2025
```
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
```
  fedfe91c
- Update docker file for supporting PD-Disaggregation on MI300x (#9494) · 988accbc
  kk authored Aug 22, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Colin Wang <kangwang@amd.com>
```
  988accbc
- chore: bump sgl-kernel v0.3.6.post2 (#9475) · b6b2287e
  Yineng Zhang authored Aug 21, 2025
  
  b6b2287e
- Add trtllm_mla and cutlass_mla for ragged fmha for chunked prefill (#9480) · 243e745d
  Elfie Guo authored Aug 21, 2025
  
  243e745d
- torch.compile() mrope (#9487) · 61a0e600
  timmy-feng authored Aug 22, 2025
  
  61a0e600
- [router] fix router load guard tracking for streaming (#9491) · 0f8cee8c
  Simo Lin authored Aug 21, 2025
  
  0f8cee8c
- [router] add tool parser base structure and partial json parser (#9482) · 816c4c85
  Chang Su authored Aug 21, 2025
  
  816c4c85
- [Docs]Update reasoning parser doc & fix outdated link (#9492) · 13ec8d42
  Xinyuan Tong authored Aug 22, 2025
```
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
```
  13ec8d42
- [docs]: fix reasoning context in docs (#9483) · 05bd7897
  Chayenne authored Aug 21, 2025
  
  05bd7897
- [code clean] add H20 cutlass groupGemm default config (#9333) · 5fd311d3
  kousakawang authored Aug 22, 2025
```
Co-authored-by: wanghanpei <wanghanpei@bytedance.com>
```
  5fd311d3
- [router] remove all tokenizer metrics for performance (#9474) · 53e2cd46
  Chang Su authored Aug 21, 2025
  
  53e2cd46
- Support MHA with chunked prefix cache for flashinfer/flashmla backend, support... · 9708d353
  Yongfei Xu authored Aug 22, 2025
```
Support MHA with chunked prefix cache for flashinfer/flashmla backend, support page size > 1 for MHA chunked prefix (#8616)
Co-authored-by: xuyongfei.xyf <xuyongfei.xyf@antgroup.com>
```
  9708d353
- [AMD] Remove the deprecated C10_WARP_SIZE (#9356) · 704ced1b
  Hubert Lu authored Aug 21, 2025
  
  704ced1b
- Add Support for Page Size greater than 1 for Flashinfer MLA Backend (#8593) · 3cc3d9b9
  Pavani Majety authored Aug 21, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  3cc3d9b9
- Update reasoning parser doc (#9468) · 0b3a5b11
  Xinyuan Tong authored Aug 22, 2025
```
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
```
  0b3a5b11
- Revert "bugfix: Fix output_ids extraction in detokenizer_manager" (#9467) · 6c855db8
  Xinyuan Tong authored Aug 22, 2025
  
  6c855db8
- feat: update auto_choose_speculative_params (#9470) · 0f9318f7
  Yineng Zhang authored Aug 21, 2025
  
  0f9318f7
- fix: tmp revert gpt oss tp sharding on hopper (#9469) · 849957bc
  Yineng Zhang authored Aug 21, 2025
  
  849957bc
21 Aug, 2025 15 commits
- [FA3] Init Spec Page Table only when Spec is enabled to save ~40MB (#9455) · cded039b
  Stefan He authored Aug 21, 2025
  
  cded039b
- feat: add fused moe config for GLM-4.5-Air-FP8 on B200 (#9463) · 275f9df3
  zixuanzhang226 authored Aug 21, 2025
  
  275f9df3
- Add deepseek v3.1 thinking parser support and update docs (#9464) · e8449ab5
  Xinyuan Tong authored Aug 22, 2025
```
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
```
  e8449ab5
- fix: support fb fp8 (#9462) · 4746aaea
  Yineng Zhang authored Aug 21, 2025
  
  4746aaea
- fix: should return a invalid request response when schema missing (#9461) · 10d34f74
  gongwei-130 authored Aug 21, 2025
  
  10d34f74
- accomendate reasoning_effort set in chat_template_kwargs (#9458) · 9ba72530
  gongwei-130 authored Aug 21, 2025
  
  9ba72530
- [5/n]decouple quantization implementation from vLLM dependency (#9454) · 9c8e4f69
  Hongbo Xu authored Aug 22, 2025
  
  9c8e4f69
- [router] add tokenizer benchmark (#9427) · 78ae1758
  Simo Lin authored Aug 21, 2025
  
  78ae1758
- [fix] Fix mxfp4 weight loading bug with TP sharding in GPT-OSS (#9433) · dae9a80f
  hlu1 authored Aug 21, 2025
```
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
```
  dae9a80f
- Fix quant kernel test errors and benchmark wrong output speeds (#7604) · e85cb1ce
  fzyzcjy authored Aug 21, 2025
  
  e85cb1ce
- Refactor weight offloading logic (#8521) · 55d336cb
  fzyzcjy authored Aug 21, 2025
  
  55d336cb
- [Bug] Fix w4afp8 moe kernel (#9392) · de4990a5
  Yuhao Yao authored Aug 21, 2025
  
  de4990a5
- ci: enhance xeon ci (#9395) · 029e0af3
  DiweiSun authored Aug 21, 2025
  
  029e0af3
- Enables speculative decoding for the trtllm_mla attention backend (#9238) · 64574ef8
  pranavm-nvidia authored Aug 21, 2025
  
  64574ef8
- [NVIDIA] Fix trtllm fp4 moe backend when used in MTP (#9384) · 18da2c96
  Kaixi Hou authored Aug 21, 2025
  
  18da2c96