Commits · 7c9697178ed3e15dbb7d4e3a66d5e52fd2caa180 · change / sglang · GitLab

28 Jul, 2025 27 commits
- [CI]Add genai-bench Performance Validation for PD Router (#8477) · 7c969717
  Keyang Ru authored Jul 28, 2025
```
Co-authored-by: key4ng <rukeyang@gamil.com>
```
  7c969717
- chore: add glm 4.5 fp8 tp4 config (#8480) · 8240a6b0
  Yineng Zhang authored Jul 28, 2025
  
  8240a6b0
- chore: add glm4 fp8 tp8 config (#8478) · 3a04aa4b
  Yineng Zhang authored Jul 28, 2025
  
  3a04aa4b
- Update codeowner (#8476) · bd516949
  Lianmin Zheng authored Jul 28, 2025
  
  bd516949
- Fix DEEPEP BF16 compatibility for Deepseek Style model like GLM 4.5 (#8469) · 74e7e457
  Stefan He authored Jul 28, 2025
```
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
```
  74e7e457
- feat: support glm4 tuning (#8473) · 1466c1b8
  Yineng Zhang authored Jul 28, 2025
  
  1466c1b8
- [3/N] MoE Refactor: Simplify DeepEP Output (#8421) · 9c138a04
  Cheng Wan authored Jul 28, 2025
  
  9c138a04
- Fix parsing ChatCompletionMessage (#7273) · c8f549d9
  Timofey authored Jul 28, 2025
```
Co-authored-by: Timofey K <timosha1113@gmail.com>
```
  c8f549d9
- [NVIDIA] Change to use `num_local_experts` (#8453) · 134fa43e
  Kaixi Hou authored Jul 28, 2025
  
  134fa43e
- fix: update dep (#8467) · ccfe52a0
  Yineng Zhang authored Jul 28, 2025
  
  ccfe52a0
- feat: throttle requests at scheduler based on --max_queued_requests (#7565) · 747dd450
  harrisonlimh authored Jul 28, 2025
  
  747dd450
- Update PR template (#8465) · b5821592
  Ke Bao authored Jul 28, 2025
  
  b5821592
- fix:reorder topk experts to ensure shared expert replaces minimal score (#8125) · a9dd3ec3
  erictanjn authored Jul 28, 2025
  
  a9dd3ec3
- chore: bump v0.4.9.post5 (#8458) · 45bc170b
  Yineng Zhang authored Jul 28, 2025
  
  45bc170b
- Revert "[kernel] opt moe align block kernel by block/warp scan algorithm" (#8457) · 22623699
  Xiaoyu Zhang authored Jul 28, 2025
  
  22623699
- Fix per_token_group_quant_8bit when hidden_dim // group_size is not divided by 4. (#8449) · fb4ce17d
  strgrb authored Jul 28, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  fb4ce17d
- fix GLM4_MOE launch with compressed_tensor quant model (#8456) · 25f73c6c
  Minglei Zhu authored Jul 28, 2025
  
  25f73c6c
- GLM-4.5 Model Support Follow-up (#8445) · 581e7dcb
  Binyao Jiang authored Jul 27, 2025
  
  581e7dcb
- doc: add bench_one_batch_server in the benchmark doc (#8441) · 484d0e02
  Qiaolin Yu authored Jul 27, 2025
  
  484d0e02
- Remove zstd compression for building Dockerfile.gb200 (#8442) · 5922c0cb
  kyleliang-nv authored Jul 27, 2025
  
  5922c0cb
- GLM-4.5 Model Support (#8224) · 6d6a8bc2
  Yuxuan Zhang authored Jul 28, 2025
```
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
```
  6d6a8bc2
- [PD] Fix abort_request for PD disaggregation (#8352) · 2fd5c704
  Shangming Cai authored Jul 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
```
  2fd5c704
- chore: bump transformer to 4.54.0 (#8416) · 4ad97370
  Stefan He authored Jul 27, 2025
```
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
```
  4ad97370
- [feat] Support different attention backends for prefill and decode (#6338) · 28103384
  Qiaolin Yu authored Jul 27, 2025
```
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  28103384
- [router] improve router logs and request id header (#8415) · fe6a445d
  Simo Lin authored Jul 27, 2025
  
  fe6a445d
- bugfix: Fix XGrammar backend to use model's EOS tokens for constrained generation (#8422) · dd487e55
  Chang Su authored Jul 27, 2025
  
  dd487e55
- Fix docker buildx push error (#8425) · bb81daef
  kyleliang-nv authored Jul 27, 2025
  
  bb81daef
27 Jul, 2025 13 commits
- Fix test_openai_server (#8419) · 58dd95fb
  Chang Su authored Jul 27, 2025
  
  58dd95fb
- bugfix: Fix multiple finish_reason chunks and tool_calls finish reason check (#8417) · b47eda33
  Chang Su authored Jul 27, 2025
  
  b47eda33
- Fix: Improve test_openai_function_calling unit test and fix... · e983d666
  Binyao Jiang authored Jul 27, 2025
```
Fix: Improve test_openai_function_calling unit test and fix reasoning_parser.py think_start_token logic (#8316)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
```
  e983d666
- Support ue8m0 for triton quant kernel (#7603) · b58c3c28
  fzyzcjy authored Jul 28, 2025
  
  b58c3c28
- Support overlapped lora updates (#8213) · df906455
  Lifu Huang authored Jul 27, 2025
  
  df906455
- Change to use native arm runner (#8414) · 95217a9b
  kyleliang-nv authored Jul 27, 2025
  
  95217a9b
- [Bugfix] Prevent PD server crash from invalid grammar (#8062) · 22e00eeb
  Shangming Cai authored Jul 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  22e00eeb
- Support triton kernels v3.4.0 for fused_moe (#8258) · b3eac168
  Yuan Luo authored Jul 27, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Cheng Wan <cwan@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  b3eac168
- chore: upgrade flashinfer v0.2.9rc2 (#8406) · 10ee8955
  Yineng Zhang authored Jul 27, 2025
  
  10ee8955
- chore: update CODEOWNERS (#8407) · bf3352c5
  Yineng Zhang authored Jul 27, 2025
  
  bf3352c5
- [hotfix] fix merge conflicts in FlashInferEPMoE (#8405) · 4d921f2b
  Cheng Wan authored Jul 27, 2025
  
  4d921f2b
- Support precomputed_embeddings for Llama 4 (#8156) · 44d600cd
  Kevin Xiang Li authored Jul 27, 2025
```
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xiang (Kevin) Li <lik@nvidia.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
```
  44d600cd
- Use FlashInfer FP4 gemm. (#8241) · 5c9c275b
  Elfie Guo authored Jul 27, 2025
  
  5c9c275b