Commits · 88f484ce4c73bff72fe417234d819800d2263d4b · change / sglang

"examples/runtime/openai_batch_complete.py" did not exist on "eba458bd19042ee449d52ea4c9f5b30a79d1165e"

02 Jul, 2025 5 commits
- Apply dsv3 router gemm kernel for deepseek-r1 fp4 (#7677) · 88f484ce
  Baizhou Zhang authored Jul 02, 2025
  
  88f484ce
- [1/n] apply wna16marlin kernel in moe weight only quantization (#7683) · 8e03b641
  AniZpZ authored Jul 02, 2025
```
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: HandH1998 <1335248067@qq.com>
Co-authored-by: 弋云 <yiyun.wyt@antgroup.com>
Co-authored-by: walker-ai <2398833647@qq.com>
```
  8e03b641
- Fix GPTQMarlinMoE (#7697) · b3fa5dc3
  Kyungmin Lee authored Jul 02, 2025
  
  b3fa5dc3
- Apply dsv3_fused_a_gemm kernel (#7635) · 00aec6ad
  Ke Bao authored Jul 02, 2025
  
  00aec6ad
- Improve error handling for requests with unloaded LoRA path(s) (#7642) · 1a08358a
  Lifu Huang authored Jul 01, 2025
  
  1a08358a
01 Jul, 2025 11 commits
- chore: upgrade flashinfer v0.2.7.post1 (#7698) · f18a8fdd
  Yineng Zhang authored Jul 01, 2025
  
  f18a8fdd
- fix(model loader): use safe_open to prevent file handle leaks. (#7684) · a7efbb27
  Simon_CQK authored Jul 02, 2025
  
  a7efbb27
- add description for llama4 eagle3 (#7688) · 93b6785d
  Yi Zhang authored Jul 01, 2025
  
  93b6785d
- upgrade sgl kernel to 0.2.1 for main (#7676) · f9eb04dd
  Zhiqiang Xie authored Jul 01, 2025
  
  f9eb04dd
- Refactor mm processors and Enable mixed modality processing (#7629) · 3a911b85
  Xinyuan Tong authored Jun 30, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  3a911b85
- support llama4 eagle3 (#6985) · 886d3449
  lukec authored Jul 01, 2025
```
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
Co-authored-by: Shenggui Li <somerlee.9@gmail.com>
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  886d3449
- chore: bump sgl-kernel v0.2.1 (#7675) · 637bfee4
  Yineng Zhang authored Jun 30, 2025
  
  637bfee4
- [CPU] remove process_group from inputs of shm_allreduce and shm_allgather (#7486) · 6005ecee
  Chunyuan WU authored Jul 01, 2025
  
  6005ecee
- Add small requirements for benchmark/parse_result tools (#7671) · ff2e9c94
  Xiaoyu Zhang authored Jul 01, 2025
  
  ff2e9c94
- Fix: sync prepare_fp8_layer_for_marlin with latest vllm changes (#7648) · 3e34e900
  narutolhy authored Jun 30, 2025
  
  3e34e900
- [doc] update lws doc for pd (#7318) · 7349717e
  ybyang authored Jul 01, 2025
  
  7349717e
30 Jun, 2025 5 commits
- chore: upgrade flashinfer v0.2.7 jit (#7663) · 392e441a
  Yineng Zhang authored Jun 30, 2025
  
  392e441a
- Add dsv3 router gemm kernel (#7627) · 7248272c
  Baizhou Zhang authored Jun 29, 2025
  
  7248272c
- Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632) · 22352d47
  Lianmin Zheng authored Jun 29, 2025
```
Co-authored-by: Kan Wu <wukanustc@gmail.com>
```
  22352d47
- [CPU] add c++ kernel to bind CPU cores and memory node (#7524) · c5131f7a
  Chunyuan WU authored Jun 30, 2025
  
  c5131f7a
- [EAGLE] remove a wrong adjustment for page_size > 1 & topk > 1 in server_args.py (#7643) · 78700893
  Lianmin Zheng authored Jun 29, 2025
  
  78700893
29 Jun, 2025 11 commits
- Update CODEOWNERS (#7640) · 663c04f7
  Lianmin Zheng authored Jun 29, 2025
  
  663c04f7
- [AMD] Add unit-test-sgl-kernel-amd to AMD CI (#7539) · 3b3f1e3a
  Hubert Lu authored Jun 29, 2025
  
  3b3f1e3a
- [misc] reduce weird rope_scaling_factor warning (#7176) · b691dcc4
  JieXin Liang authored Jun 30, 2025
  
  b691dcc4
- Move files related to EPLB (#7580) · 0c9c6c75
  fzyzcjy authored Jun 30, 2025
  
  0c9c6c75
- [bugfix] fix runtime dropping panic in editable (#7628) · e3f9b548
  Simo Lin authored Jun 29, 2025
  
  e3f9b548
- Fix sgl-router startup crash (#7619) · b3cff365
  finetune authored Jun 29, 2025
  
  b3cff365
- Fix stream reasoning parser and Adds Kimi reasoning parser (#7432) · 8f335b5b
  Xinyuan Tong authored Jun 29, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  8f335b5b
- Add @mickqian as the CODEOWNERS of multimodal (#7636) · b2264076
  Lianmin Zheng authored Jun 29, 2025
  
  b2264076
- Add dsv3 fused a gemm to sgl-kernel (#7630) · 04b35190
  Ke Bao authored Jun 29, 2025
  
  04b35190
- [Minor] clean up multimodal processor and tokenizer manager (#7624) · 071a1f51
  Lianmin Zheng authored Jun 29, 2025
  
  071a1f51
- [bugfix] Remove PR comment posting from Rust benchmark workflow (#7625) · 7c0db3a6
  Simo Lin authored Jun 28, 2025
  
  7c0db3a6
28 Jun, 2025 8 commits
- oai: Adds support for OpenAI chat completions API in bench_serving (#7036) · c45e49d8
  Xinyuan Tong authored Jun 28, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
```
  c45e49d8
- docs: add gb200 nvl72 and a16z grant (#7620) · d8053929
  Yineng Zhang authored Jun 28, 2025
  
  d8053929
- Let EP prefill support new DeepGEMM (#7310) · 00c7b1ad
  fzyzcjy authored Jun 28, 2025
  
  00c7b1ad
- Let ep_scatter support arbitrary strides / ue8m0 format (#7309) · 82eccae4
  fzyzcjy authored Jun 28, 2025
  
  82eccae4
- fix unit tests (#7618) · a8c10aee
  Yineng Zhang authored Jun 28, 2025
  
  a8c10aee
- [PD] Respect sampling_params.max_new_tokens when PD disaggregation is activated (#7598) · eb429b88
  Shangming Cai authored Jun 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  eb429b88
- Support dynamic LoRA loading / unloading in engine/server API (#7446) · 49538d11
  Lifu Huang authored Jun 27, 2025
  
  49538d11
- [BUG] fix local_rank in initialize_dp_attention (#7584) · cfe2edac
  Sheng Qi authored Jun 28, 2025
  
  cfe2edac