Commits · eebfdb94593ffdf92ac7bac3dbc82c80a3baf3f2 · change / sglang · GitLab

27 Apr, 2025 5 commits
- [fix] fix potential bumpy throughtput with deepgemm (#5722) · eebfdb94
  JieXin Liang authored Apr 27, 2025
  
  eebfdb94
- Use device_id in dist init to reduce NCCL communicator warmup & creation overhead (#5728) · dfb32264
  Wenxuan Tan authored Apr 26, 2025
  
  dfb32264
- fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512 (#5733) · 63c13a2c
  Kyungmin Lee authored Apr 27, 2025
  
  63c13a2c
- Add an assertion to enhance the robustness of the operator (#5736) · 4d1e52ab
  liwenju0 authored Apr 27, 2025
  
  4d1e52ab
- [Minor] fix documentations (#5756) · 155890e4
  Lianmin Zheng authored Apr 26, 2025
  
  155890e4
26 Apr, 2025 4 commits
- Bugfix for minicpmo vision test (#5760) · 1f963d7f
  Yi Zhang authored Apr 26, 2025
  
  1f963d7f
- [Fix]: support deepseek-vl2-tiny model (#5552) · 04d0123f
  ZXN authored Apr 26, 2025
```
Co-authored-by: bppps <zouyu.zzx@alibaba-inc.com>
```
  04d0123f
- fix: fix one more bug from merging mm_inputs (#5718) · feda9b11
  Mick authored Apr 26, 2025
```
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: XinyuanTong <115166877+JustinTong0323@users.noreply.github.com>
```
  feda9b11
- Reorder loop in shared expert weight loading (#5719) · c3948ba6
  Ke Bao authored Apr 26, 2025
  
  c3948ba6
25 Apr, 2025 11 commits
- [Docs] Update runtime/engine/readme.md (#5737) · 269c457e
  Michael Yao authored Apr 26, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  269c457e
- update triton 3.2.0 h200 fused moe triton config and add warning about triton... · 18ce468d
  Xiaoyu Zhang authored Apr 26, 2025
```
update triton 3.2.0 h200 fused moe triton config and add warning about triton fused_moe_kernel performance degradation due to different Triton versions. (#5740)
```
  18ce468d
- Disable flaky eagle tests (#5753) · 21514ff5
  Lianmin Zheng authored Apr 25, 2025
  
  21514ff5
- Revert "[Model] Support `ArcticForCausalLM` architecture... · 5641a094
  Lianmin Zheng authored Apr 25, 2025
```
Revert "[Model] Support `ArcticForCausalLM` architecture (Snowflake/snowflake-arctic-instruct)" (#5754)
```
  5641a094
- Pin torch audio to 2.6.0 (#5750) · 3dd3538c
  Lianmin Zheng authored Apr 25, 2025
  
  3dd3538c
- Fix: deepseek forward absorb (#5723) · 93c6fb12
  michael-amd authored Apr 25, 2025
```
Co-authored-by: ispobock <ispobaoke@163.com>
```
  93c6fb12
- [PD]: Support Muti Prefill in one node (#5704) · 11e27d09
  IAN authored Apr 26, 2025
```
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  11e27d09
- [PD] Add kvargs table and thread pool for kvcache sender of mooncake (#5738) · 50eda839
  shangmingc authored Apr 25, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  50eda839
- [PD] Better logs (#5715) · c55550cb
  Liangsheng Yin authored Apr 25, 2025
  
  c55550cb
- [Model] Support `ArcticForCausalLM` architecture (Snowflake/snowflake-arctic-instruct) (#5078) · 43fb95c2
  Brayden Zhong authored Apr 25, 2025
```
Co-authored-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
```
  43fb95c2
- Add MMMU benchmark results (#4491) · 7d9679b7
  Ravi Theja authored Apr 25, 2025
```
Co-authored-by: Ravi Theja Desetty <ravitheja@Ravis-MacBook-Pro.local>
```
  7d9679b7
24 Apr, 2025 5 commits
- [Doc] Fix a link to Weilin Zhao (#5706) · b5be5694
  Michael Yao authored Apr 25, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  b5be5694
- Add example to use sgl engine with fastapi (#5648) · d2b8d0b8
  Ravi Theja authored Apr 24, 2025
```
Co-authored-by: Ravi Theja Desetty <ravitheja@Ravis-MacBook-Pro.local>
```
  d2b8d0b8
- Fix weight loading bug for Deepseek v3+nextn (#5684) · a14654dd
  Baizhou Zhang authored Apr 24, 2025
  
  a14654dd
- [BugFix] Fix combination of MTP and `--n-share-experts-fusion`with R1 (#5707) · 5d93a950
  Yuhong Guo authored Apr 24, 2025
  
  5d93a950
- vlm: enable radix cache for qwen-vl models (#5349) · c998d04b
  Mick authored Apr 24, 2025
```
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
```
  c998d04b
23 Apr, 2025 14 commits
- chore: bump sgl-kernel 0.1.0 (#5688) · 7d0edf3c
  Yineng Zhang authored Apr 23, 2025
  
  7d0edf3c
- fix: only compile ApplyTokenBitmaskInplace cu124+ (#5686) · ce4ecba4
  Yineng Zhang authored Apr 23, 2025
  
  ce4ecba4
- fix: update truss bench_serving (#5683) · b1f6d89b
  Yineng Zhang authored Apr 23, 2025
  
  b1f6d89b
- [Doc] Fix two 404 links caused by sglang typo (#5667) · 7c99103f
  Michael Yao authored Apr 23, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  7c99103f
- tune the threshold of gemma-2-27b-it in test_nightly_gsm8k_eval.py (#5677) · de071366
  Lianmin Zheng authored Apr 23, 2025
  
  de071366
- [PD] Add support for dp attention with mooncake (#5530) · e0673969
  shangmingc authored Apr 23, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  e0673969
- fix torchvision::nms not exist (#5671) · 127ff898
  Yineng Zhang authored Apr 23, 2025
  
  127ff898
- fix gemma3 unit test (#5670) · 8777a1d2
  Yineng Zhang authored Apr 23, 2025
  
  8777a1d2
- Integrating PD disaggregation with DP attention and DeepEP (#5435) · 711efe78
  Cheng Wan authored Apr 23, 2025
```
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  711efe78
- fix awq_dequantize import (#5669) · fbb5f229
  Yineng Zhang authored Apr 23, 2025
  
  fbb5f229
- fix sgl-kernel unit tests (#5666) · 15fabcc0
  Yineng Zhang authored Apr 23, 2025
  
  15fabcc0
- [1/2] Add FP8 Blockscale MoE CUTLASS kernel for Blackwell (#5281) · e62c4955
  Elfie Guo authored Apr 22, 2025
  
  e62c4955
- Remove unnecessary `torch.full` in DeepSeek (#5601) · 71d1785f
  fzyzcjy authored Apr 23, 2025
  
  71d1785f
- Fuse q_a_proj and kv_a_proj (#5619) · 3f87f831
  Baizhou Zhang authored Apr 22, 2025
  
  3f87f831
22 Apr, 2025 1 commit
- Turn on DeepGemm By Default and Update Doc (#5628) · ce5412b6
  Baizhou Zhang authored Apr 22, 2025
  
  ce5412b6