Commits · 22630ca24266f4e2837bf0a02b6f899edb575601 · change / sglang · GitLab

30 May, 2025 5 commits
- Support sliding window in triton backend (#6509) · 22630ca2
  Jianan Ji authored May 30, 2025
  
  22630ca2
- Fix aiohttp 'Chunk too big' in bench_serving (#6737) · d279d499
  Yuhong Guo authored May 30, 2025
  
  d279d499
- [PD] Optimize time out logic and add env var doc for mooncake (#6761) · 6cb00c63
  shangmingc authored May 30, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  6cb00c63
- Update DeepSeek-R1-0528 function call chat template (#6765) · 62cac2c4
  Xu Wenqing authored May 30, 2025
```
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
```
  62cac2c4
- Improve EPLB logical to physical dispatch map (#6727) · 2c3b71d6
  fzyzcjy authored May 30, 2025
  
  2c3b71d6
29 May, 2025 17 commits
- [fix][RL] Fix DeepSeekV3ForCausalLM.post_load_weights for multiple update weight (#6265) · 51cdd81f
  Zilin Zhu authored May 30, 2025
  
  51cdd81f
- Fix mem_fraction_static for AMD CI (#6748) · 73def253
  Baizhou Zhang authored May 29, 2025
  
  73def253
- [test] add ut and bm for get_last_loc (#6746) · d9d35def
  JieXin Liang authored May 30, 2025
  
  d9d35def
- Support tuning DeepEP configs (#6742) · 6df81e8a
  fzyzcjy authored May 29, 2025
  
  6df81e8a
- Support picking variants of EPLB algorithms (#6728) · 3ab7d9b5
  fzyzcjy authored May 29, 2025
  
  3ab7d9b5
- Super tiny enable sole usage of expert distribution metrics and update doc (#6680) · 7e5071c9
  fzyzcjy authored May 29, 2025
  
  7e5071c9
- PD Rust LB (PO2) (#6437) · 78689d33
  Liangsheng Yin authored May 29, 2025
```
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
```
  78689d33
- [PD] Support completion endpoint (#6729) · 1dc6864f
  shangmingc authored May 29, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  1dc6864f
- refactor apply_w8a8_block_fp8_linear in fp (#6545) · 485a023b
  ChangyiYang authored May 29, 2025
  
  485a023b
- Add draft extend CUDA graph for Triton backend (#6705) · 7e412900
  Ke Bao authored May 29, 2025
  
  7e412900
- refactor(tool call): Fix BaseFormatDetector tool_index issue and refactor... · c673727e
  Chang Su authored May 29, 2025
```
refactor(tool call): Fix BaseFormatDetector tool_index issue and refactor `parse_streaming_increment` (#6715)
```
  c673727e
- Add DeepSeek-R1-0528 function call chat template (#6725) · f4d4f939
  Xu Wenqing authored May 29, 2025
```
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
```
  f4d4f939
- Tune memory arguments on B200 (#6718) · f2bd3515
  Baizhou Zhang authored May 29, 2025
  
  f2bd3515
- [PD] bug fix: Update status if nixl receiver send a a dummy req. (#6720) · c459536b
  dongmao zhang authored May 29, 2025
  
  c459536b
- [fix] more mem for draft_extend cuda_graph (#6726) · 535c8386
  JieXin Liang authored May 29, 2025
  
  535c8386
- [feat] triton kernel for get_last_loc (#6676) · 2163586e
  JieXin Liang authored May 29, 2025
  
  2163586e
- Fix PP for Qwen3 MoE (#6709) · e06b0761
  iLeGend authored May 29, 2025
  
  e06b0761
28 May, 2025 18 commits
- Fix LoRA bench (#6719) · 844a8f42
  Wenxuan Tan authored May 28, 2025
  
  844a8f42
- [Feature] Support Flashinfer fp8 blockwise GEMM kernel on Blackwell (#6479) · 791b3bfa
  Baizhou Zhang authored May 28, 2025
  
  791b3bfa
- Speed up when having padding tokens two-batch overlap (#6668) · 31589e17
  fzyzcjy authored May 29, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  31589e17
- Minor refactor two-batch overlap (#6682) · ae6a5b29
  fzyzcjy authored May 29, 2025
  
  ae6a5b29
- Overlap two kernels in DeepSeek with communication (#6711) · 4839999b
  fzyzcjy authored May 29, 2025
  
  4839999b
- Fuse routed_scaling_factor in DeepSeek (#6710) · 541a985f
  fzyzcjy authored May 29, 2025
  
  541a985f
- [PD] Remove Unnecessary Exception Handling for FastQueue.get() (#6712) · 5170b010
  Hongbo Xu authored May 29, 2025
  
  5170b010
- [CI] Fix setup of disaggregation with different tp (#6706) · d63e76f7
  shangmingc authored May 29, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  d63e76f7
- [Bugfix] Fix ChatCompletion endpoint of mini_lb when stream is set (#6703) · e9fd11c0
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  e9fd11c0
- [Bugfix] Fix slice operation when chunk size mismatch (#6697) · c7588d59
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  c7588d59
- [PD Perf] replace Queue to FastQueue (#6649) · 6b231325
  ybyang authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  6b231325
- [PD] Abort unbootstrapped prefill requests through timeout (#6685) · b1c8d4e9
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  b1c8d4e9
- [CI] Fix flaky pp single node test (#6689) · c25231c6
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  c25231c6
- [Bugfix] Fix missing abort finish reason for PD with ChatCompletion (#6693) · fba03b29
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  fba03b29
- fix(deepseekv3): Fix DeepSeekV3Detector tool_index assignment and multi-tool... · 461a7302
  Chang Su authored May 28, 2025
```
fix(deepseekv3): Fix DeepSeekV3Detector tool_index assignment and multi-tool call streaming support (#6655)
```
  461a7302
- fix log_info_on_rank0 error when run benchmark (#6260) · 07610353
  Xiaoyu Zhang authored May 28, 2025
  
  07610353
- Refine pre_reorder_triton_kernel slightly to improve performance (#6627) · c087ddd6
  Yuan Luo authored May 28, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
```
  c087ddd6
- Update amd docker and nightly models. (#6687) · f4a8987f
  Sai Enduri authored May 28, 2025
  
  f4a8987f