Commits · 3abc30364d4f4ef2aa0e5382bc9a52111e48e11a · change / sglang · GitLab

25 Jun, 2025 7 commits
- [ci] add router benchmark script and CI (#7498) · 3abc3036
  Simo Lin authored Jun 25, 2025
  
  3abc3036
- clean duplicate code (#7512) · afeed465
  linzhuo authored Jun 25, 2025
  
  afeed465
- EPLB support for MTP (#7510) · 587b4c6e
  yilian49 authored Jun 25, 2025
  
  587b4c6e
- [PD][Spec] Fix hidden state transfer for spec decode (#7516) · 7b9a174a
  Shangming Cai authored Jun 25, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  7b9a174a
- [OAI] patch origin request_id logic (#7508) · 03c039c4
  ybyang authored Jun 25, 2025
  
  03c039c4
- Fuse sorted_token_ids padding to moe_align_block_size kernel (#7437) · 57ab7769
  Ke Bao authored Jun 25, 2025
  
  57ab7769
- misc: Improvement to serving_chat.py and add more ut (#7489) · 112b496a
  Chang Su authored Jun 24, 2025
  
  112b496a
24 Jun, 2025 14 commits
- fix: Add `--model` as an alias for `--model-path` in server_args (#7505) · 3562256b
  Chang Su authored Jun 24, 2025
  
  3562256b
- [PD] NIXL: Register kv args in advance and cleanup finished requests (#6717) · 5f527834
  Trevor Morris authored Jun 24, 2025
  
  5f527834
- Support multi-thread model weight loading (#7277) · 9f1787fa
  xianzhiT authored Jun 25, 2025
  
  9f1787fa
- [benchmark] fbgemm benchmark support bandwidth report and support fbgemm_cutlass_gmm (#7422) · 8ecad0b1
  Xiaoyu Zhang authored Jun 25, 2025
  
  8ecad0b1
- Remove cumsum_buffer initilization (#7439) · 7151194b
  Ke Bao authored Jun 24, 2025
  
  7151194b
- [PD Disaggregation] replace transfer with batch transfer for better performance (#7236) · 2ed68d7a
  Francis authored Jun 24, 2025
  
  2ed68d7a
- enable aiter_biased_grouped_topk kernel (#7423) · e984d507
  valarLip authored Jun 24, 2025
  
  e984d507
- [AMD] add aiter fused moe in DeepEP path (#7268) · 755f3147
  Alex Sun authored Jun 24, 2025
  
  755f3147
- chore: bump v0.4.8 (#7493) · 7c3a12c0
  Yineng Zhang authored Jun 23, 2025
  
  7c3a12c0
- chore: bump sgl-kernel v0.2.0 (#7490) · e846d95e
  Yineng Zhang authored Jun 23, 2025
  
  e846d95e
- Fix MTP with Deepseek R1 Fp4 (#7376) · 15f34013
  Charles Chen authored Jun 23, 2025
  
  15f34013
- Fix RequestValidationError response format (#7487) · d04163b3
  Chang Su authored Jun 23, 2025
  
  d04163b3
- bugfix: Prevent global mutation of conv.stop_str across requests (#7347) · 7732bbe4
  huangtingwei authored Jun 24, 2025
```
Co-authored-by: Chang Su <chang.s.su@oracle.com>
```
  7732bbe4
- Perormance: Enable cuda graph for dp idle batch (#7269) · ed0a0b69
  u4lr451 authored Jun 24, 2025
```
Co-authored-by: austindeng <austindeng@tencent.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: ch-wan <cwan39@gatech.edu>
```
  ed0a0b69
23 Jun, 2025 17 commits
- ci: Revert openai_server related tests in AMD suites (#7449) · fa42e419
  Chang Su authored Jun 23, 2025
  
  fa42e419
- Support weight loading without mmap (#7469) · e5afb88b
  Yuhong Guo authored Jun 24, 2025
  
  e5afb88b
- Quick fix for DeepGemm requant to also cover MTP. (#7378) · e5ddeb04
  Charles Chen authored Jun 23, 2025
  
  e5ddeb04
- [perf] slightly imporve DeepSeek-R1-FP4 TP8 (#7481) · bdbb8d00
  JieXin Liang authored Jun 24, 2025
  
  bdbb8d00
- kvcache io kernels and test case (#7382) · 34c3f9b2
  Zhiqiang Xie authored Jun 23, 2025
  
  34c3f9b2
- update mooncake in dockerfile (#7480) · 76139bfb
  Liangsheng Yin authored Jun 24, 2025
  
  76139bfb
- Fix dtype for idle input in spec decoding (#7456) · f8d48fd3
  Cheng Wan authored Jun 23, 2025
  
  f8d48fd3
- feat(func_call): Add more check in `BaseFormatDetector.parse_streaming_increment` (#7479) · 34b6b842
  Chang Su authored Jun 23, 2025
  
  34b6b842
- Fix prefill OOM due to wrong token calculation when page > 1 (#7397) · 25549433
  Liangsheng Yin authored Jun 24, 2025
  
  25549433
- [CI] Upgrade mooncake to 0.3.4.post1 to fix 8 gpu tests (#7472) · d6dddc19
  Shangming Cai authored Jun 24, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  d6dddc19
- Fix a bug in BatchTokenIDOut & Misc style and dependency updates (#7457) · 55e03b10
  Lianmin Zheng authored Jun 23, 2025
  
  55e03b10
- Solve docker build failed in the virtual machine (#7290) · 8aa68ed5
  kk authored Jun 23, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
```
  8aa68ed5
- feat: integrate deepgemm into EPMoE (#6821) · 506c4928
  xutizhou authored Jun 23, 2025
```
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: TianQiLin666666 <1834987979@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  506c4928
- Update hyperparameter_tuning.md (#7454) · 30ceccc7
  Lianmin Zheng authored Jun 22, 2025
  
  30ceccc7
- Fix CUDA Graph Check under Deepep with DP FFN (#7451) · ac5010e0
  Cheng Wan authored Jun 22, 2025
  
  ac5010e0
- add fused moe config for qwen3 in triton3.3.1 (#7445) · 3cee035e
  Yi Zhang authored Jun 23, 2025
  
  3cee035e
- [misc] Add PD service discovery support in router (#7361) · 30f2a44a
  Simo Lin authored Jun 22, 2025
  
  30f2a44a
22 Jun, 2025 2 commits
- Fix torch compile run (#7391) · bd4f5818
  kk authored Jun 23, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
```
  bd4f5818
- Remove copy after bmm (#7441) · 50f1b6d6
  Ke Bao authored Jun 23, 2025
  
  50f1b6d6