Commits · f6f96b0521ca87b61199c9ff262ceafa1f3c0f81 · change / sglang · GitLab

08 May, 2025 14 commits
- [AMD] switch to custom allreduce regardless of MSCCL setting on ROCm (#6097) · 2a936a84
  Hubert Lu authored May 08, 2025
  
  2a936a84
- [perf] dsv3 bmm fallback to bf16 (#5662) · 5e023301
  JieXin Liang authored May 09, 2025
  
  5e023301
- [Feature] Add FlashAttention3 as a backend for VisionAttention (#5764) · fa7d7fd9
  Zhu Chen authored May 09, 2025
```
Co-authored-by: othame <chenzhu_912@zju.edu.cn>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yi Zhang <1109276519@qq.com>
```
  fa7d7fd9
- [fix] fix pyproject.toml dependencies (#6119) · f1ff736d
  JieXin Liang authored May 08, 2025
  
  f1ff736d
- DeepEP normal support deepgemm-contiguous (#5626) · acc816d8
  lukec authored May 08, 2025
```
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
Co-authored-by: ZhengHSI <zhenghsi@qq.com>
```
  acc816d8
- [PD] Add control to slow down a server (#5572) · cef91b1e
  fzyzcjy authored May 08, 2025
  
  cef91b1e
- Tiny refactor weight loading logic (#5232) · 6450c122
  fzyzcjy authored May 08, 2025
  
  6450c122
- Tiny refactor ModelConfig.from_server_args (#5219) · b6cf3532
  fzyzcjy authored May 08, 2025
  
  b6cf3532
- Overlap shared expert and routed expert computations (#5121) · 3b2680a4
  fzyzcjy authored May 08, 2025
  
  3b2680a4
- optimize pad operations in fa3 to accelarate 100+us (#6077) · 79961afa
  Minglei Zhu authored May 07, 2025
  
  79961afa
- adding Triton configs for DeepSeekV3 FusedMoE kernel on Blackwell (#6111) · cfca4e0e
  Baizhou Zhang authored May 07, 2025
  
  cfca4e0e
- [CI]Add performance CI for VLM (#6038) · e88dd482
  XinyuanTong authored May 07, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  e88dd482
- Clean logs for DeepSeek-V3 launching (#6079) · 73600673
  Baizhou Zhang authored May 07, 2025
  
  73600673
- [refactor] slightly tidy fp8 module (#5993) · b70957fc
  JieXin Liang authored May 08, 2025
  
  b70957fc
07 May, 2025 5 commits
- feat(engine): add bootstrap parameters to generate methods (dynamo) (#6075) · e444c13f
  ishandhanani authored May 07, 2025
  
  e444c13f
- Super tiny fix doc (#5233) · c68de479
  fzyzcjy authored May 07, 2025
  
  c68de479
- Hint users DeepEP normal mode is incompatible with CUDA Graph (#5014) · 4c7b4242
  fzyzcjy authored May 07, 2025
  
  4c7b4242
- [Feature] Support for Ascend NPU backend (#3853) · 00c2c1f0
  Song Zhang authored May 07, 2025
```
Signed-off-by: Song Zhang <gepin.zs@antgroup.com>
Co-authored-by: 22dimensions <waitingwind@foxmail.com>
```
  00c2c1f0
- Add DeepEP to CI PR Test (#5655) · 8a828666
  Jinyan Chen authored May 07, 2025
```
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
```
  8a828666
06 May, 2025 3 commits
- Better PD initialization (#5751) · a3e4e9bf
  Liangsheng Yin authored May 07, 2025
  
  a3e4e9bf
- Fix not "import os" (#6057) · 6d4d3bc8
  Liangsheng Yin authored May 06, 2025
  
  6d4d3bc8
- Fix problem of large page size with chunked prefill (#6046) · b26cb1c5
  Zhiqiang Xie authored May 06, 2025
  
  b26cb1c5
05 May, 2025 7 commits
- Fix prefill OOM error in the case of large page size (#5081) · f8e46093
  Zhiqiang Xie authored May 05, 2025
  
  f8e46093
- [Security][Bug] Prevent binding to all TCP interfaces (#5752) · 683707c3
  Adarsh Shirawalmath authored May 06, 2025
  
  683707c3
- feat: append more comprehensive fields in messages instead of merely role and content (#5996) · a68ed766
  mlmz authored May 06, 2025
  
  a68ed766
- feat: Add a unified merge_state API (#5428) · 82653f66
  DefTruth authored May 06, 2025
  
  82653f66
- Fix "Avoid computing lse in Ragged Prefill when there's no prefix match" (#5555) · 22da3d97
  Wenxuan Tan authored May 05, 2025
  
  22da3d97
- [PD] Optimize disaggregation ib device help info (#5781) · 56f6589e
  shangmingc authored May 05, 2025
  
  56f6589e
- [PD] Allow customizing reserved tokens to avoid KV cache waste (#6002) · 3008db9c
  fzyzcjy authored May 05, 2025
  
  3008db9c
04 May, 2025 4 commits
- fix: fix broadcast_pyobj breaking VerlEngine (#5997) · 357fb2db
  Junrong Lin authored May 05, 2025
  
  357fb2db
- Fix flaky issues of lora and add multi batch tests (#5957) · 3042f1da
  Qiaolin Yu authored May 04, 2025
  
  3042f1da
- [Minor] Fix duplicate method definitions in conversation.py (#6012) · 2b63798c
  Lifu Huang authored May 04, 2025
```
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
```
  2b63798c
- [Fix] Suppress dynamo logging when using flashinfer backend with torch compile (#5992) · bf203cb7
  Baizhou Zhang authored May 04, 2025
  
  bf203cb7
03 May, 2025 2 commits
- [perf] H100 DeepSeek-V3 fused moe tuned config (#5998) · 8ebde73f
  JieXin Liang authored May 04, 2025
  
  8ebde73f
- Fix Phi3 serving which was broke by earlier change (#5991) · 6b0fae79
  Stefan He authored May 03, 2025
```
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
```
  6b0fae79
02 May, 2025 5 commits
- Overlap qk norm with two streams (#5977) · d8ab6011
  Ke Bao authored May 03, 2025
  
  d8ab6011
- Fix set kv cache multi-stream (#5975) · 6579cd7d
  Ke Bao authored May 03, 2025
  
  6579cd7d
- [PD] NIXL backend Prefill TP & Decode TP+DP (#5681) · 97ac42b6
  Yongtong Wu authored May 02, 2025
  
  97ac42b6
- FA3 speed up: skip len operation and get batch size directly from forward batch (#5969) · 1acca3a2
  Lifu Huang authored May 02, 2025
```
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
```
  1acca3a2
- Support InternVL3 (#5350) · 3409aaab
  xm:D authored May 02, 2025
```
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  3409aaab