Commits · f4e3ebeb056002fdb2649cba1f3758522516e81a · change / sglang

23 Sep, 2025 34 commits
- [router] Support streaming for Openai Router Response api (#10822) · f4e3ebeb
  Keyang Ru authored Sep 23, 2025
  
  f4e3ebeb
- [Auto Sync] Update simple_eval_common.py (20250923) (#10824) · 312bfc4c
  Lianmin Zheng authored Sep 23, 2025
```
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
```
  312bfc4c
- [Auto Sync] Update elementwise.py (20250923) (#10823) · e290303e
  Lianmin Zheng authored Sep 23, 2025
```
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  e290303e
- fix: draft model IMA by overide max_positional_embeddings (#10787) · aab35bcc
  Xinyuan Tong authored Sep 23, 2025
```
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
```
  aab35bcc
- [Auto Sync] Update protocol.py (20250923) (#10820) · 42aedb02
  Yineng Zhang authored Sep 23, 2025
```
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cicirori <32845984+cicirori@users.noreply.github.com>
```
  42aedb02
- add tunning files for QWEN-3-NEXT (#10794) · 984730b7
  Yiakwy authored Sep 24, 2025
  
  984730b7
- Fix latest main ci (#10799) · 23632d35
  Shangming Cai authored Sep 24, 2025
```
Signed-off-by: Shangming Cai <csmthu@gmail.com>
```
  23632d35
- [router] fix axum default body limit (#10818) · 08b8c0c3
  Chang Su authored Sep 23, 2025
  
  08b8c0c3
- Remove duplicate code in qwen2 model (#10540) · d42975c6
  Lzhang-hub authored Sep 24, 2025
  
  d42975c6
- fix ceval (#10504) · adc24a3a
  ZhengHSI authored Sep 24, 2025
```
Co-authored-by: lukec <118525388+sleepcoo@users.noreply.github.com>
```
  adc24a3a
- router(grpc): Implement route for chat_cmpl endpoint (#10761) · 7ff93e61
  Chang Su authored Sep 23, 2025
  
  7ff93e61
- [router] use dashmap for radix tree instead of hash for multi model (#10814) · b24b2e7e
  Simo Lin authored Sep 23, 2025
  
  b24b2e7e
- [ROCm] Update aiter to v0.1.5.post3 (#10812) · 7135db5d
  sogalin authored Sep 24, 2025
  
  7135db5d
- [fix][pd-disag]no need set next batch sampling info done in prefill (#10259) · 4b5ef300
  Jimmy authored Sep 24, 2025
  
  4b5ef300
- model: support qwen3-vl series (#10323) · 4f564b9e
  Zheng Li authored Sep 24, 2025
```
Co-authored-by: ocss884 <ocss.lin@gmail.com>
Co-authored-by: cao1zhg <653506626@qq.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: 瑀澈 <yuche.lz@alibaba-inc.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  4f564b9e
- [router] responses api POST and GET with local storage (#10581) · 98c3b04f
  Simo Lin authored Sep 23, 2025
```
Co-authored-by: key4ng <rukeyang@gmail.com>
```
  98c3b04f
- [router] fix cache aware routing strategy and lock contention (#10773) · ddab4fc7
  Simo Lin authored Sep 23, 2025
  
  ddab4fc7
- Fix hicache mooncake backend CI (#10792) · d21c3522
  Shangming Cai authored Sep 23, 2025
```
Signed-off-by: Shangming Cai <csmthu@gmail.com>
```
  d21c3522
- move `environ` into `sglang.srt` to avoid break SRT auto sync. (#10791) · 4a762041
  Liangsheng Yin authored Sep 23, 2025
  
  4a762041
- Clean up server args (#10770) · ea338676
  Lianmin Zheng authored Sep 23, 2025
  
  ea338676
- followup: clean up dockerfiles and release yamls (#10783) · b06db198
  ishandhanani authored Sep 23, 2025
  
  b06db198
- chore: upgrade sgl-kernel 0.3.12 (#10782) · 8c1ef0f9
  Yineng Zhang authored Sep 23, 2025
  
  8c1ef0f9
- Introduce `FutureMap` (#10715) · f5a2faf2
  Liangsheng Yin authored Sep 23, 2025
  
  f5a2faf2
- feat: unify dockerfiles (#10705) · 1c82d9db
  ishandhanani authored Sep 23, 2025
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  1c82d9db
- Move cached kernel to srt.utils (#10776) · 9241f4fd
  Lifu Huang authored Sep 22, 2025
  
  9241f4fd
- Fix trtllm_mla slow concat kernel in MTP (#10777) · 063c3791
  fzyzcjy authored Sep 23, 2025
  
  063c3791
- Use simulate acc len from `sglang.environ` (#10771) · 632b7d8c
  Liangsheng Yin authored Sep 23, 2025
  
  632b7d8c
- [router] fix logger type mismatch (#10774) · 16adf3dc
  Chang Su authored Sep 22, 2025
```
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
```
  16adf3dc
- [router] remove pd router draining channel (#10767) · c3a1d775
  Simo Lin authored Sep 22, 2025
  
  c3a1d775
- [router] refactor router and worker management 4/n (#10756) · 89971c4c
  Simo Lin authored Sep 22, 2025
```
Co-authored-by: Chang Su <chang.s.su@oracle.com>
```
  89971c4c
- [Auto Sync] Update configurer.py (20250923) (#10765) · 113f8f65
  Lianmin Zheng authored Sep 22, 2025
```
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
```
  113f8f65
- [Ascend]optimize Qwen3 on Ascend (#10574) · e22f3a5e
  ronnie_zheng authored Sep 23, 2025
```
Co-authored-by: c30031083 <chenxu140@huawei.com>
```
  e22f3a5e
- [Ascend] optimize Qwen-vl on Ascend (#10556) · 095093ee
  ronnie_zheng authored Sep 23, 2025
```
Co-authored-by: wangqihui01 <wangqh10@163.com>
```
  095093ee
- [Feature] Add MLAProcess for DeepSeek MLA on NPU (#10130) · d27a6f70
  Even Zhou authored Sep 23, 2025
  
  d27a6f70
22 Sep, 2025 6 commits

[Auto Sync] Update flashattention_backend.py (20250922) (#10762) · 0753ef83

Yineng Zhang authored Sep 22, 2025


Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gordon Gustafson <ggustafson@together.ai>

0753ef83

fix: kv events with tp > 1 (#10541) · 662393f2
ishandhanani authored Sep 23, 2025

662393f2

Enables TRT-LLM backend to be used for target_verify (#10281) · b1bb8e74

pranavm-nvidia authored Sep 22, 2025


Co-authored-by: Pranav Marathe <pranavm@ipp1-3309.ipp1a1.colossus.nvidia.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>

b1bb8e74

Fix multimodal registry and code sync scripts (#10759) · 38c00ed7
Lianmin Zheng authored Sep 22, 2025
```
Co-authored-by: cctry <shiyang@x.ai>
```
38c00ed7

refactor zero copy (#10300) · d4041a5e

pansicheng authored Sep 23, 2025

Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>

d4041a5e

[Generative Score API] Added test_scores_api.py to github CICD to run per commit (#10755) · 2f555c4c
Vedant Jhaveri authored Sep 22, 2025
```
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
Co-authored-by: Sundara Raman Ramachandran <sundar24295@gmail.com>
```
2f555c4c