Commits · 6ae3f05b337b13c381b7215bfb4a9fce44c39b1f · change / sglang · GitLab

03 Oct, 2025 13 commits
- Fix CUDA illegal memory access issues in speculative decoding (#10892) · 6ae3f05b
  ur4t authored Oct 03, 2025
  
  6ae3f05b
- Tiny move files to utils folder (#11166) · fdc4e1e5
  fzyzcjy authored Oct 03, 2025
  
  fdc4e1e5
- [hot-fix] Fix CI break which caused by adding `thinking_mode` in eval (#11192) · 04b86b3c
  Liangsheng Yin authored Oct 03, 2025
  
  04b86b3c
- Add --thinking-mode to run_eval (#11189) · d6777a70
  hlu1 authored Oct 03, 2025
```
Signed-off-by: Hao Lu <14827759+hlu1@users.noreply.github.com>
```
  d6777a70
- [Feature] Option to save model weights to CPU when memory saver mode is enabled (#10873) · 8c574902
  Matt Nappo authored Oct 03, 2025
```
Co-authored-by: molocule <34072934+molocule@users.noreply.github.com>
```
  8c574902
- [router] Steaming support for MCP Tool Calls in OpenAI Router (#11173) · 34151f17
  Keyang Ru authored Oct 03, 2025
  
  34151f17
- Tiny add PD disaggregation + DP attention test (#11167) · 6794d210
  fzyzcjy authored Oct 03, 2025
  
  6794d210
- fix: radix cache memory accounting (#10637) · 1a31229c
  Alex Chi Z authored Oct 03, 2025
```
Signed-off-by: Alex Chi Z <iskyzh@gmail.com>
```
  1a31229c
- [CI]] Tee server logs to both file and stdout/stderr using PIPE (#11185) · de89ef49
  Liangsheng Yin authored Oct 03, 2025
  
  de89ef49
- [Fix] Update to v0.1.5.post4 and refine HIP attention backend selection (#11161) · b00a0c78
  jacky.cheng authored Oct 03, 2025
  
  b00a0c78
- [1/n] Enable DCA CUDA graph capture (#9537) · a2faf894
  b8zhong authored Oct 02, 2025
  
  a2faf894
- [Generative Scores API] add performance tests to CICD (#10830) · 7e61737d
  Vedant V Jhaveri authored Oct 02, 2025
  
  7e61737d
- Introduce naming convention in `io_struct` and base sglang io classes. (#10133) · 3c699772
  Liangsheng Yin authored Oct 03, 2025
  
  3c699772
02 Oct, 2025 27 commits
- Allow use of TRTLLM_MHA backend for hybrid attention on Blackwell (#11138) · e8100774
  Dom Brown authored Oct 03, 2025
  
  e8100774
- [router][grpc] Support streaming for v1/chat/completions (#11179) · 963175d5
  Chang Su authored Oct 02, 2025
  
  963175d5
- fix: shoudn't include CUDA_ARCH 100 and 120 for cuda12.6.1 (#11176) · 0618ad6d
  gongwei-130 authored Oct 02, 2025
  
  0618ad6d
- Minor fixes for server_args, parallel_state, and test_deterministic.py (#11159) · 6a261aac
  Lianmin Zheng authored Oct 02, 2025
  
  6a261aac
- Remove dp balance metadata and minimul token balance. (#11170) · 7ff740a6
  Liangsheng Yin authored Oct 03, 2025
  
  7ff740a6
- [grpc] style fix for grpc compilation. (#11175) · bfcd9b24
  Liangsheng Yin authored Oct 03, 2025
  
  bfcd9b24
- Unify forward output datastructure (#11124) · 458611de
  Liangsheng Yin authored Oct 03, 2025
  
  458611de
- [proto] Add script to compile python protos (#11171) · 3511b370
  Chang Su authored Oct 02, 2025
  
  3511b370
- Tiny remove duplicated code (#11164) · afcd3e10
  fzyzcjy authored Oct 02, 2025
  
  afcd3e10
- Tiny fix ep_gather behavior different in CI (#11130) · 12d68183
  fzyzcjy authored Oct 02, 2025
  
  12d68183
- Tiny cleanup deepseek_v2.py (#11163) · b65db028
  fzyzcjy authored Oct 02, 2025
  
  b65db028
- fix cpp JIT compilation issue of ngram speculative decoding (#10837) · 948278f1
  b8zhong authored Oct 02, 2025
  
  948278f1
- Clean up ascend allocator (#11152) · 7d004799
  Liangsheng Yin authored Oct 02, 2025
  
  7d004799
- [model] Add mamba2 and Falcon-H1 support. (#10988) · 083629c2
  ilyasch2 authored Oct 02, 2025
```
Co-authored-by: Younes Belkada <younes.belkada@tii.ae>
Co-authored-by: Younes B <49240599+younesbelkada@users.noreply.github.com>
```
  083629c2
- [router][grpc] Support tool call parser in streaming (#11160) · b658be6f
  Chang Su authored Oct 02, 2025
  
  b658be6f
- Support single batch overlap (#10422) · 5e786cca
  fzyzcjy authored Oct 02, 2025
  
  5e786cca
- Support dispatch low latency (#10263) · 0b9dfba7
  fzyzcjy authored Oct 02, 2025
```
Co-authored-by: Kaixi Hou <4001424+kaixih@users.noreply.github.com>
```
  0b9dfba7
- Remove unused pack `.item()` in paged allocator. (#11156) · 6a290034
  Liangsheng Yin authored Oct 02, 2025
  
  6a290034
- Tiny detect slow ranks (#10508) · 2ac453b0
  fzyzcjy authored Oct 02, 2025
  
  2ac453b0
- Fuse quantize and rope in trtllm_mla MTP (#10779) · f35def86
  fzyzcjy authored Oct 02, 2025
  
  f35def86
- Tiny fix missing alt stream in nextn layer (#10768) · d61615fe
  fzyzcjy authored Oct 02, 2025
  
  d61615fe
- Tiny improve dumper (#11132) · b1ccaf01
  fzyzcjy authored Oct 02, 2025
  
  b1ccaf01
- Clean up parallel_state.py (#11148) · 097725bb
  Lianmin Zheng authored Oct 02, 2025
  
  097725bb
- Fix DeepSeek chunked prefill memory issue (#11149) · 44b1fbe2
  fzyzcjy authored Oct 02, 2025
  
  44b1fbe2
- [ROCm] To reduce the compiling time when using torch compile. (#10559) · c0dbbdd1
  sogalin authored Oct 02, 2025
  
  c0dbbdd1
- Fix ngram spec with page size > 1 (#11135) · 25e7dbe8
  Liangsheng Yin authored Oct 02, 2025
  
  25e7dbe8
- Intoduce cpu tensor as metadata to avoid blocking gpu kernel launch (#10720) · 0b2aa8a7
  Zhang Junda authored Oct 02, 2025
```
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
```
  0b2aa8a7