Commits · 8ba313304d53e9975462e18b9b52f1b46c72802e · change / sglang · GitLab

27 Apr, 2025 24 commits
- Revert "fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512" (#5772) · 8ba31330
  Lianmin Zheng authored Apr 26, 2025
  
  8ba31330
- add switch to disable open api doc (#3744) · 02102063
  zhanweidu authored Apr 27, 2025
```
Signed-off-by: congcongke <zhanweidu@163.com>
```
  02102063
- Add memory_saver check (#4986) · 7e944246
  Kebe authored Apr 27, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  7e944246
- Use sgl-kernel sgl_per_token_group_quant_int8 (#4971) · a086a113
  lambert0312 authored Apr 27, 2025
  
  a086a113
- update llguidance to 0.7.11; adds StructTag (#4870) · bdbe5f81
  Michał Moskal authored Apr 26, 2025
  
  bdbe5f81
- fix(srt): check if sample_indices is not None before usage. (#5633) · 9ad28f63
  aoshen524 authored Apr 26, 2025
  
  9ad28f63
- Handle JSONDecodeError while processing request data (#5599) · d7b1ce65
  yan97ao authored Apr 27, 2025
  
  d7b1ce65
- [misc] more decode step log for batch_one_batch (#5565) · f55933e1
  JieXin Liang authored Apr 27, 2025
  
  f55933e1
- Add Llama 4 to FA3 test (#5509) · 408ba022
  Stefan He authored Apr 26, 2025
  
  408ba022
- fix: Use `is not None` instead of `!= None` for None checks. (#5687) · 094891c0
  vzed authored Apr 26, 2025
  
  094891c0
- support for the DeepSeek model by enabling streaming response parsing (#5592) · a21ef363
  Frankey_8080 authored Apr 27, 2025
  
  a21ef363
- [fix] fix bench_one_batch_server (#5607) · 3c4dc38a
  JieXin Liang authored Apr 27, 2025
  
  3c4dc38a
- [feature] support for roberta embedding models (#5730) · d8fbc7c0
  DavidBao authored Apr 27, 2025
  
  d8fbc7c0
- Update amd docker image to `sglang:v0.4.5.post3-rocm630`. (#5697) · c5e1026f
  saienduri authored Apr 26, 2025
  
  c5e1026f
- Fuse MLA set kv cache kernel (#5748) · 799c4bb5
  Ke Bao authored Apr 27, 2025
  
  799c4bb5
- CI: rewrite test_vision_chunked_prefill to speedup (#5682) · 02723e1b
  Mick authored Apr 27, 2025
  
  02723e1b
- we fix the non existent access of `decrypted_config_file` (#5685) · df2cf583
  vzed authored Apr 26, 2025
  
  df2cf583
- perf: update H20 fused_moe_triton kernel config to get higher throughput during prefilling (#5716) · 133ded03
  saltyfish66 authored Apr 27, 2025
  
  133ded03
- Resolves the `404 Not Found` error when running `compile_deep_gemm.py` in multi-node setups (#5720) · f87a6ab3
  Yuhong Guo authored Apr 27, 2025
  
  f87a6ab3
- [fix] fix potential bumpy throughtput with deepgemm (#5722) · eebfdb94
  JieXin Liang authored Apr 27, 2025
  
  eebfdb94
- Use device_id in dist init to reduce NCCL communicator warmup & creation overhead (#5728) · dfb32264
  Wenxuan Tan authored Apr 26, 2025
  
  dfb32264
- fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512 (#5733) · 63c13a2c
  Kyungmin Lee authored Apr 27, 2025
  
  63c13a2c
- Add an assertion to enhance the robustness of the operator (#5736) · 4d1e52ab
  liwenju0 authored Apr 27, 2025
  
  4d1e52ab
- [Minor] fix documentations (#5756) · 155890e4
  Lianmin Zheng authored Apr 26, 2025
  
  155890e4
26 Apr, 2025 4 commits
- Bugfix for minicpmo vision test (#5760) · 1f963d7f
  Yi Zhang authored Apr 26, 2025
  
  1f963d7f
- [Fix]: support deepseek-vl2-tiny model (#5552) · 04d0123f
  ZXN authored Apr 26, 2025
```
Co-authored-by: bppps <zouyu.zzx@alibaba-inc.com>
```
  04d0123f
- fix: fix one more bug from merging mm_inputs (#5718) · feda9b11
  Mick authored Apr 26, 2025
```
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: XinyuanTong <115166877+JustinTong0323@users.noreply.github.com>
```
  feda9b11
- Reorder loop in shared expert weight loading (#5719) · c3948ba6
  Ke Bao authored Apr 26, 2025
  
  c3948ba6
25 Apr, 2025 11 commits
- [Docs] Update runtime/engine/readme.md (#5737) · 269c457e
  Michael Yao authored Apr 26, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  269c457e
- update triton 3.2.0 h200 fused moe triton config and add warning about triton... · 18ce468d
  Xiaoyu Zhang authored Apr 26, 2025
```
update triton 3.2.0 h200 fused moe triton config and add warning about triton fused_moe_kernel performance degradation due to different Triton versions. (#5740)
```
  18ce468d
- Disable flaky eagle tests (#5753) · 21514ff5
  Lianmin Zheng authored Apr 25, 2025
  
  21514ff5
- Revert "[Model] Support `ArcticForCausalLM` architecture... · 5641a094
  Lianmin Zheng authored Apr 25, 2025
```
Revert "[Model] Support `ArcticForCausalLM` architecture (Snowflake/snowflake-arctic-instruct)" (#5754)
```
  5641a094
- Pin torch audio to 2.6.0 (#5750) · 3dd3538c
  Lianmin Zheng authored Apr 25, 2025
  
  3dd3538c
- Fix: deepseek forward absorb (#5723) · 93c6fb12
  michael-amd authored Apr 25, 2025
```
Co-authored-by: ispobock <ispobaoke@163.com>
```
  93c6fb12
- [PD]: Support Muti Prefill in one node (#5704) · 11e27d09
  IAN authored Apr 26, 2025
```
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  11e27d09
- [PD] Add kvargs table and thread pool for kvcache sender of mooncake (#5738) · 50eda839
  shangmingc authored Apr 25, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  50eda839
- [PD] Better logs (#5715) · c55550cb
  Liangsheng Yin authored Apr 25, 2025
  
  c55550cb
- [Model] Support `ArcticForCausalLM` architecture (Snowflake/snowflake-arctic-instruct) (#5078) · 43fb95c2
  Brayden Zhong authored Apr 25, 2025
```
Co-authored-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
```
  43fb95c2
- Add MMMU benchmark results (#4491) · 7d9679b7
  Ravi Theja authored Apr 25, 2025
```
Co-authored-by: Ravi Theja Desetty <ravitheja@Ravis-MacBook-Pro.local>
```
  7d9679b7
24 Apr, 2025 1 commit
- [Doc] Fix a link to Weilin Zhao (#5706) · b5be5694
  Michael Yao authored Apr 25, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  b5be5694