Commits · d9d35def3db5ebfd02e26173458e207b2732ecfd · change / sglang · GitLab

29 May, 2025 15 commits
- [test] add ut and bm for get_last_loc (#6746) · d9d35def
  JieXin Liang authored May 30, 2025
  
  d9d35def
- Support tuning DeepEP configs (#6742) · 6df81e8a
  fzyzcjy authored May 29, 2025
  
  6df81e8a
- Support picking variants of EPLB algorithms (#6728) · 3ab7d9b5
  fzyzcjy authored May 29, 2025
  
  3ab7d9b5
- Super tiny enable sole usage of expert distribution metrics and update doc (#6680) · 7e5071c9
  fzyzcjy authored May 29, 2025
  
  7e5071c9
- PD Rust LB (PO2) (#6437) · 78689d33
  Liangsheng Yin authored May 29, 2025
```
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
```
  78689d33
- [PD] Support completion endpoint (#6729) · 1dc6864f
  shangmingc authored May 29, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  1dc6864f
- refactor apply_w8a8_block_fp8_linear in fp (#6545) · 485a023b
  ChangyiYang authored May 29, 2025
  
  485a023b
- Add draft extend CUDA graph for Triton backend (#6705) · 7e412900
  Ke Bao authored May 29, 2025
  
  7e412900
- refactor(tool call): Fix BaseFormatDetector tool_index issue and refactor... · c673727e
  Chang Su authored May 29, 2025
```
refactor(tool call): Fix BaseFormatDetector tool_index issue and refactor `parse_streaming_increment` (#6715)
```
  c673727e
- Add DeepSeek-R1-0528 function call chat template (#6725) · f4d4f939
  Xu Wenqing authored May 29, 2025
```
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
```
  f4d4f939
- Tune memory arguments on B200 (#6718) · f2bd3515
  Baizhou Zhang authored May 29, 2025
  
  f2bd3515
- [PD] bug fix: Update status if nixl receiver send a a dummy req. (#6720) · c459536b
  dongmao zhang authored May 29, 2025
  
  c459536b
- [fix] more mem for draft_extend cuda_graph (#6726) · 535c8386
  JieXin Liang authored May 29, 2025
  
  535c8386
- [feat] triton kernel for get_last_loc (#6676) · 2163586e
  JieXin Liang authored May 29, 2025
  
  2163586e
- Fix PP for Qwen3 MoE (#6709) · e06b0761
  iLeGend authored May 29, 2025
  
  e06b0761
28 May, 2025 18 commits
- Fix LoRA bench (#6719) · 844a8f42
  Wenxuan Tan authored May 28, 2025
  
  844a8f42
- [Feature] Support Flashinfer fp8 blockwise GEMM kernel on Blackwell (#6479) · 791b3bfa
  Baizhou Zhang authored May 28, 2025
  
  791b3bfa
- Speed up when having padding tokens two-batch overlap (#6668) · 31589e17
  fzyzcjy authored May 29, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  31589e17
- Minor refactor two-batch overlap (#6682) · ae6a5b29
  fzyzcjy authored May 29, 2025
  
  ae6a5b29
- Overlap two kernels in DeepSeek with communication (#6711) · 4839999b
  fzyzcjy authored May 29, 2025
  
  4839999b
- Fuse routed_scaling_factor in DeepSeek (#6710) · 541a985f
  fzyzcjy authored May 29, 2025
  
  541a985f
- [PD] Remove Unnecessary Exception Handling for FastQueue.get() (#6712) · 5170b010
  Hongbo Xu authored May 29, 2025
  
  5170b010
- [CI] Fix setup of disaggregation with different tp (#6706) · d63e76f7
  shangmingc authored May 29, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  d63e76f7
- [Bugfix] Fix ChatCompletion endpoint of mini_lb when stream is set (#6703) · e9fd11c0
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  e9fd11c0
- [Bugfix] Fix slice operation when chunk size mismatch (#6697) · c7588d59
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  c7588d59
- [PD Perf] replace Queue to FastQueue (#6649) · 6b231325
  ybyang authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  6b231325
- [PD] Abort unbootstrapped prefill requests through timeout (#6685) · b1c8d4e9
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  b1c8d4e9
- [CI] Fix flaky pp single node test (#6689) · c25231c6
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  c25231c6
- [Bugfix] Fix missing abort finish reason for PD with ChatCompletion (#6693) · fba03b29
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  fba03b29
- fix(deepseekv3): Fix DeepSeekV3Detector tool_index assignment and multi-tool... · 461a7302
  Chang Su authored May 28, 2025
```
fix(deepseekv3): Fix DeepSeekV3Detector tool_index assignment and multi-tool call streaming support (#6655)
```
  461a7302
- fix log_info_on_rank0 error when run benchmark (#6260) · 07610353
  Xiaoyu Zhang authored May 28, 2025
  
  07610353
- Refine pre_reorder_triton_kernel slightly to improve performance (#6627) · c087ddd6
  Yuan Luo authored May 28, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
```
  c087ddd6
- Update amd docker and nightly models. (#6687) · f4a8987f
  Sai Enduri authored May 28, 2025
  
  f4a8987f
27 May, 2025 7 commits
- feat: Add warnings for invalid tool_choice and UTs (#6582) · 41ba767f
  Chang Su authored May 27, 2025
  
  41ba767f
- Add batch test for draft extend (#6672) · f127355a
  Ke Bao authored May 28, 2025
  
  f127355a
- fix(tool call): Fix tool_index in PythonicDetector and issues with mixed... · bdb962d7
  Chang Su authored May 27, 2025
```
fix(tool call): Fix tool_index in PythonicDetector and issues with mixed output in non-streaming (#6678)
```
  bdb962d7
- Disable compiling arch below sm_90 in aarch64 by default (#6380) · 0b9557fc
  Qiaolin Yu authored May 27, 2025
  
  0b9557fc
- Support gathering expert distribution details (#6665) · 87068b5c
  fzyzcjy authored May 28, 2025
  
  87068b5c
- Fix DeepEP error in Qwen 3 MoE models (#6673) · a564e001
  fzyzcjy authored May 28, 2025
  
  a564e001
- [CI] update verlengine ci to 4-gpu test (#6007) · 2103b806
  Junrong Lin authored May 28, 2025
  
  2103b806