Commits · 8675bdf2461550a19192c334a46d55c58f314dbf · change / sglang

03 Aug, 2025 1 commit
- Support limiting max loaded loras in CPU. (#8650) · 8675bdf2
  Lifu Huang authored Aug 03, 2025
  
  8675bdf2
01 Aug, 2025 3 commits
- Add support for NCCL symmetric memory for TP allreduces (#8238) · 82e6c3a6
  Nicolas Castet authored Aug 01, 2025
  
  82e6c3a6
- fix arg typo for --disaggregation-transfer-backend (#8664) · b17c5b01
  Zac authored Aug 02, 2025
  
  b17c5b01
- [5/N] MoE Refactor: Update MoE parallelism arguments (#8658) · 6c88f6c8
  Cheng Wan authored Aug 01, 2025
  
  6c88f6c8
31 Jul, 2025 2 commits

TRTLLM Gen MLA Decode Kernel Integration (same as #7938) (#8632) · 4b04998d
Faraz authored Jul 31, 2025
```
Signed-off-by: Faraz Khoubsirat <58580514+farazkh80@users.noreply.github.com>
```
4b04998d

model: support Step3V (#8583) · 51c38163

Chang Su authored Jul 31, 2025


Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: nnnobody-code <nnnobody@foxmail.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Qiaolin-Yu <qy254@cornell.edu>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>

51c38163

28 Jul, 2025 2 commits
- [NVIDIA] Change to use `num_local_experts` (#8453) · 134fa43e
  Kaixi Hou authored Jul 28, 2025
  
  134fa43e
- [feat] Support different attention backends for prefill and decode (#6338) · 28103384
  Qiaolin Yu authored Jul 27, 2025
```
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  28103384
27 Jul, 2025 1 commit

Support precomputed_embeddings for Llama 4 (#8156) · 44d600cd

Kevin Xiang Li authored Jul 27, 2025


Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xiang (Kevin) Li <lik@nvidia.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>

44d600cd

25 Jul, 2025 1 commit
- [Feat] Add reasoning parser for Qwen/Qwen3-235B-A22B-Thinking-2507 (#8363) · d8ee1564
  Chang Su authored Jul 25, 2025
  
  d8ee1564
21 Jul, 2025 1 commit
- [Refactor] simplify multimodal data processing (#8107) · 8430bfe3
  Xinyuan Tong authored Jul 20, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  8430bfe3
19 Jul, 2025 2 commits
- Support start up LoRA server without initial adapters (#8019) · 4e3defe5
  Lifu Huang authored Jul 19, 2025
  
  4e3defe5
- Clean up server args (#8161) · bb0e8a32
  Lianmin Zheng authored Jul 19, 2025
  
  bb0e8a32
18 Jul, 2025 1 commit
- [Doc] Steps to add a new attention backend (#8155) · 9c7a4618
  Lianmin Zheng authored Jul 18, 2025
  
  9c7a4618
14 Jul, 2025 1 commit
- Refactor dynamic LoRA update to fix incorrect handling of variant weight shapes (#7844) · e2ed9d04
  Lifu Huang authored Jul 13, 2025
  
  e2ed9d04
11 Jul, 2025 2 commits
- [feature] kv transfer support of ascend npu (#7795) · 86044712
  ronnie_zheng authored Jul 11, 2025
```
Co-authored-by: liupeng <liupeng374@huawei.com>
```
  86044712
- Support Kimi K2 (#7940) · 61555307
  Atream authored Jul 11, 2025
  
  61555307
09 Jul, 2025 2 commits
- Update native_api doc to match the change in the `get_model_info` endpoint (#7660) · 08702321
  Yikai Zhang authored Jul 09, 2025
```
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
```
  08702321
- [PD] Add guidance for prefill bootstrap timeout (#7846) · 64c5907e
  Shangming Cai authored Jul 09, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  64c5907e
03 Jul, 2025 2 commits
- fix CI: update native api ipynb (#7754) · 43f93f63
  Xinyuan Tong authored Jul 03, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  43f93f63
- Ascend attention backend(PA&MLA) (#7722) · 1e0e5497
  ronnie_zheng authored Jul 03, 2025
```
Co-authored-by: Maksim <makcum888e@mail.ru>
Co-authored-by: VDV1985 <vladdv85@mail.ru>
```
  1e0e5497
30 Jun, 2025 1 commit
- Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632) · 22352d47
  Lianmin Zheng authored Jun 29, 2025
```
Co-authored-by: Kan Wu <wukanustc@gmail.com>
```
  22352d47
26 Jun, 2025 1 commit
- [PD] Raise error for incompatible mooncake version and some minor fixes (#7527) · 5c214257
  Shangming Cai authored Jun 26, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  5c214257
23 Jun, 2025 1 commit
- Update hyperparameter_tuning.md (#7454) · 30ceccc7
  Lianmin Zheng authored Jun 22, 2025
  
  30ceccc7
21 Jun, 2025 2 commits
- feat(oai refactor): Replace `openai_api` with `entrypoints/openai` (#7351) · 72676cd6
  Chang Su authored Jun 21, 2025
```
Co-authored-by: Jin Pan <jpan236@wisc.edu>
```
  72676cd6
- Remove batches api in docs & example (#7400) · ab74f8f0
  Jinn authored Jun 20, 2025
  
  ab74f8f0
20 Jun, 2025 1 commit
- [Doc] add embedding rerank doc (#7364) · 97011abc
  woodx authored Jun 20, 2025
  
  97011abc
17 Jun, 2025 1 commit
- support qwen2 running on ascend npu device (#7022) · a39d9287
  Yijie Zhu authored Jun 18, 2025
```
Co-authored-by: 刁莹煜 <diaoyingyu1@hisilicon.com>
```
  a39d9287
16 Jun, 2025 1 commit
- Minor style and doc fix (#7228) · 21615cc3
  Lianmin Zheng authored Jun 16, 2025
  
  21615cc3
12 Jun, 2025 1 commit
- [Fix] Reduce busy polling when scheduler is idle (#6026) · bd7cfbd2
  Povilas Kanapickas authored Jun 13, 2025
  
  bd7cfbd2
11 Jun, 2025 2 commits
- Clean up docs for server args and sampling parameters (generated by grok) (#7076) · dbdf76ca
  Lianmin Zheng authored Jun 10, 2025
  
  dbdf76ca
- update doc (#7046) · f2a75a66
  Ximingwang-09 authored Jun 11, 2025
```
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
```
  f2a75a66
10 Jun, 2025 2 commits
- Improve perf tuning docs (#7071) · 90bd3e32
  Lianmin Zheng authored Jun 10, 2025
  
  90bd3e32
- Open AI API hidden states (#6716) · b56de8f9
  kyle-pena-kuzco authored Jun 10, 2025
  
  b56de8f9
05 Jun, 2025 2 commits
- [PD] Fix potential perf spike caused by tracker gc and optimize doc (#6764) · dd1012fc
  shangmingc authored Jun 06, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  dd1012fc
- support 1 shot allreduce in 1-node and 2-node using mscclpp (#6277) · 8e3797be
  zyksir authored Jun 05, 2025
  
  8e3797be
04 Jun, 2025 2 commits
- [Refactor] Multimodal data processing for VLM (#6659) · cf9815ba
  Xinyuan Tong authored Jun 04, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  cf9815ba
- [FEAT] Add transformers backend support (#5929) · 37f15475
  Marc Sun authored Jun 04, 2025
  
  37f15475
31 May, 2025 1 commit
- Improve profiler and integrate profiler in bench_one_batch_server (#6787) · 2d72fc47
  Lianmin Zheng authored May 31, 2025
  
  2d72fc47
30 May, 2025 1 commit
- [PD] Optimize time out logic and add env var doc for mooncake (#6761) · 6cb00c63
  shangmingc authored May 30, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  6cb00c63