Commits · 844a8f42c74bcd917e9f1456d406ba4f1deebda3 · change / sglang · GitLab

28 May, 2025 18 commits
- Fix LoRA bench (#6719) · 844a8f42
  Wenxuan Tan authored May 28, 2025
  
  844a8f42
- [Feature] Support Flashinfer fp8 blockwise GEMM kernel on Blackwell (#6479) · 791b3bfa
  Baizhou Zhang authored May 28, 2025
  
  791b3bfa
- Speed up when having padding tokens two-batch overlap (#6668) · 31589e17
  fzyzcjy authored May 29, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  31589e17
- Minor refactor two-batch overlap (#6682) · ae6a5b29
  fzyzcjy authored May 29, 2025
  
  ae6a5b29
- Overlap two kernels in DeepSeek with communication (#6711) · 4839999b
  fzyzcjy authored May 29, 2025
  
  4839999b
- Fuse routed_scaling_factor in DeepSeek (#6710) · 541a985f
  fzyzcjy authored May 29, 2025
  
  541a985f
- [PD] Remove Unnecessary Exception Handling for FastQueue.get() (#6712) · 5170b010
  Hongbo Xu authored May 29, 2025
  
  5170b010
- [CI] Fix setup of disaggregation with different tp (#6706) · d63e76f7
  shangmingc authored May 29, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  d63e76f7
- [Bugfix] Fix ChatCompletion endpoint of mini_lb when stream is set (#6703) · e9fd11c0
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  e9fd11c0
- [Bugfix] Fix slice operation when chunk size mismatch (#6697) · c7588d59
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  c7588d59
- [PD Perf] replace Queue to FastQueue (#6649) · 6b231325
  ybyang authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  6b231325
- [PD] Abort unbootstrapped prefill requests through timeout (#6685) · b1c8d4e9
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  b1c8d4e9
- [CI] Fix flaky pp single node test (#6689) · c25231c6
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  c25231c6
- [Bugfix] Fix missing abort finish reason for PD with ChatCompletion (#6693) · fba03b29
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  fba03b29
- fix(deepseekv3): Fix DeepSeekV3Detector tool_index assignment and multi-tool... · 461a7302
  Chang Su authored May 28, 2025
```
fix(deepseekv3): Fix DeepSeekV3Detector tool_index assignment and multi-tool call streaming support (#6655)
```
  461a7302
- fix log_info_on_rank0 error when run benchmark (#6260) · 07610353
  Xiaoyu Zhang authored May 28, 2025
  
  07610353
- Refine pre_reorder_triton_kernel slightly to improve performance (#6627) · c087ddd6
  Yuan Luo authored May 28, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
```
  c087ddd6
- Update amd docker and nightly models. (#6687) · f4a8987f
  Sai Enduri authored May 28, 2025
  
  f4a8987f
27 May, 2025 22 commits
- feat: Add warnings for invalid tool_choice and UTs (#6582) · 41ba767f
  Chang Su authored May 27, 2025
  
  41ba767f
- Add batch test for draft extend (#6672) · f127355a
  Ke Bao authored May 28, 2025
  
  f127355a
- fix(tool call): Fix tool_index in PythonicDetector and issues with mixed... · bdb962d7
  Chang Su authored May 27, 2025
```
fix(tool call): Fix tool_index in PythonicDetector and issues with mixed output in non-streaming (#6678)
```
  bdb962d7
- Disable compiling arch below sm_90 in aarch64 by default (#6380) · 0b9557fc
  Qiaolin Yu authored May 27, 2025
  
  0b9557fc
- Support gathering expert distribution details (#6665) · 87068b5c
  fzyzcjy authored May 28, 2025
  
  87068b5c
- Fix DeepEP error in Qwen 3 MoE models (#6673) · a564e001
  fzyzcjy authored May 28, 2025
  
  a564e001
- [CI] update verlengine ci to 4-gpu test (#6007) · 2103b806
  Junrong Lin authored May 28, 2025
  
  2103b806
- [PD] Make bootstrap code common between NIXL and Mooncake (#6473) · e806f708
  Trevor Morris authored May 27, 2025
  
  e806f708
- Revert "fix communicator for non-dp lm head (#6662)" (#6677) · fa6723f0
  Yineng Zhang authored May 27, 2025
  
  fa6723f0
- Speed up expert location update (#6661) · 673ff668
  fzyzcjy authored May 28, 2025
  
  673ff668
- Fix OOM when updating expert locations (#6660) · 447be242
  fzyzcjy authored May 28, 2025
  
  447be242
- DeepSeek: enable none block-quant FP8 quantizations (#6638) · 183d9f96
  HAI authored May 27, 2025
  
  183d9f96
- Support EAGLE draft extend CUDA graph (#6606) · 63195028
  Ke Bao authored May 27, 2025
```
Co-authored-by: Sehoon Kim <sehoonkim@berkeley.edu>
```
  63195028
- fix communicator for non-dp lm head (#6662) · a3d7f4b6
  Cheng Wan authored May 27, 2025
  
  a3d7f4b6
- Fix qwen3 tbo/dp-lm-head (#6652) · b18416fb
  Yi Zhang authored May 27, 2025
  
  b18416fb
- fix: fix nightly test from updating transformers (#6658) · ce9d690e
  Mick authored May 27, 2025
  
  ce9d690e
- Add environment flag for disabling message queue broadcaster (#6403) · bdaefbbf
  Baizhou Zhang authored May 26, 2025
  
  bdaefbbf
- docs: Update documentation to reflect xgrammar as default grammar backend (#6601) · 45a31a82
  Vincent Zhong authored May 27, 2025
```
Co-authored-by: b8zhong <b8zhong@uwaterloo.ca>
```
  45a31a82
- Add note to add supported model to documentation (#6640) · 1aa0fbf4
  Brayden Zhong authored May 27, 2025
  
  1aa0fbf4
- update toc for doc and dockerfile code style format (#6450) · 7a0bbe6a
  linzhuo authored May 27, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  7a0bbe6a
- [Bugfix]: Fix call for function_call_parser.multi_format_detector in adapter.py (#6650) · ae335842
  Chang Su authored May 26, 2025
  
  ae335842
- Refactor LoRA handling to support adapter tensors in fused format (#6585) · 477a101c
  Lifu Huang authored May 26, 2025
  
  477a101c