Commits · f9fb33efc356869c2955ad236fa4a54aa5b471c3 · change / sglang · GitLab

27 Apr, 2025 28 commits
- [CI] Fix test case (#5790) · a38f6932
  Lianmin Zheng authored Apr 27, 2025
  
  a38f6932
- [PD]Reduce kv transfer threads (#5791) · beb65c74
  Liangsheng Yin authored Apr 27, 2025
  
  beb65c74
- [CI] Fix ci tests (#5769) · 621e96bf
  Lianmin Zheng authored Apr 27, 2025
  
  621e96bf
- [CI] fix port conflicts (#5789) · 35ca04d2
  Lianmin Zheng authored Apr 27, 2025
  
  35ca04d2
- Revert "Use device_id in dist init to reduce NCCL communicator warmup & creation overhead" (#5786) · 9c088829
  Lianmin Zheng authored Apr 27, 2025
  
  9c088829
- Revert "[fix] fix bench_one_batch_server" (#5785) · 005aad32
  Lianmin Zheng authored Apr 27, 2025
  
  005aad32
- Revert "Revert "fix: import vllm_rotary_embedding error when head_size not in... · 6e313c1b
  Lianmin Zheng authored Apr 27, 2025
```
Revert "Revert "fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512"" (#5777)
```
  6e313c1b
- Revert "fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512" (#5772) · 8ba31330
  Lianmin Zheng authored Apr 26, 2025
  
  8ba31330
- add switch to disable open api doc (#3744) · 02102063
  zhanweidu authored Apr 27, 2025
```
Signed-off-by: congcongke <zhanweidu@163.com>
```
  02102063
- Add memory_saver check (#4986) · 7e944246
  Kebe authored Apr 27, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  7e944246
- Use sgl-kernel sgl_per_token_group_quant_int8 (#4971) · a086a113
  lambert0312 authored Apr 27, 2025
  
  a086a113
- update llguidance to 0.7.11; adds StructTag (#4870) · bdbe5f81
  Michał Moskal authored Apr 26, 2025
  
  bdbe5f81
- fix(srt): check if sample_indices is not None before usage. (#5633) · 9ad28f63
  aoshen524 authored Apr 26, 2025
  
  9ad28f63
- Handle JSONDecodeError while processing request data (#5599) · d7b1ce65
  yan97ao authored Apr 27, 2025
  
  d7b1ce65
- [misc] more decode step log for batch_one_batch (#5565) · f55933e1
  JieXin Liang authored Apr 27, 2025
  
  f55933e1
- Add Llama 4 to FA3 test (#5509) · 408ba022
  Stefan He authored Apr 26, 2025
  
  408ba022
- fix: Use `is not None` instead of `!= None` for None checks. (#5687) · 094891c0
  vzed authored Apr 26, 2025
  
  094891c0
- support for the DeepSeek model by enabling streaming response parsing (#5592) · a21ef363
  Frankey_8080 authored Apr 27, 2025
  
  a21ef363
- [fix] fix bench_one_batch_server (#5607) · 3c4dc38a
  JieXin Liang authored Apr 27, 2025
  
  3c4dc38a
- [feature] support for roberta embedding models (#5730) · d8fbc7c0
  DavidBao authored Apr 27, 2025
  
  d8fbc7c0
- Fuse MLA set kv cache kernel (#5748) · 799c4bb5
  Ke Bao authored Apr 27, 2025
  
  799c4bb5
- we fix the non existent access of `decrypted_config_file` (#5685) · df2cf583
  vzed authored Apr 26, 2025
  
  df2cf583
- perf: update H20 fused_moe_triton kernel config to get higher throughput during prefilling (#5716) · 133ded03
  saltyfish66 authored Apr 27, 2025
  
  133ded03
- Resolves the `404 Not Found` error when running `compile_deep_gemm.py` in multi-node setups (#5720) · f87a6ab3
  Yuhong Guo authored Apr 27, 2025
  
  f87a6ab3
- [fix] fix potential bumpy throughtput with deepgemm (#5722) · eebfdb94
  JieXin Liang authored Apr 27, 2025
  
  eebfdb94
- Use device_id in dist init to reduce NCCL communicator warmup & creation overhead (#5728) · dfb32264
  Wenxuan Tan authored Apr 26, 2025
  
  dfb32264
- fix: import vllm_rotary_embedding error when head_size not in 64, 128, 256, 512 (#5733) · 63c13a2c
  Kyungmin Lee authored Apr 27, 2025
  
  63c13a2c
- Add an assertion to enhance the robustness of the operator (#5736) · 4d1e52ab
  liwenju0 authored Apr 27, 2025
  
  4d1e52ab
26 Apr, 2025 4 commits
- Bugfix for minicpmo vision test (#5760) · 1f963d7f
  Yi Zhang authored Apr 26, 2025
  
  1f963d7f
- [Fix]: support deepseek-vl2-tiny model (#5552) · 04d0123f
  ZXN authored Apr 26, 2025
```
Co-authored-by: bppps <zouyu.zzx@alibaba-inc.com>
```
  04d0123f
- fix: fix one more bug from merging mm_inputs (#5718) · feda9b11
  Mick authored Apr 26, 2025
```
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: XinyuanTong <115166877+JustinTong0323@users.noreply.github.com>
```
  feda9b11
- Reorder loop in shared expert weight loading (#5719) · c3948ba6
  Ke Bao authored Apr 26, 2025
  
  c3948ba6
25 Apr, 2025 8 commits
- update triton 3.2.0 h200 fused moe triton config and add warning about triton... · 18ce468d
  Xiaoyu Zhang authored Apr 26, 2025
```
update triton 3.2.0 h200 fused moe triton config and add warning about triton fused_moe_kernel performance degradation due to different Triton versions. (#5740)
```
  18ce468d
- Disable flaky eagle tests (#5753) · 21514ff5
  Lianmin Zheng authored Apr 25, 2025
  
  21514ff5
- Revert "[Model] Support `ArcticForCausalLM` architecture... · 5641a094
  Lianmin Zheng authored Apr 25, 2025
```
Revert "[Model] Support `ArcticForCausalLM` architecture (Snowflake/snowflake-arctic-instruct)" (#5754)
```
  5641a094
- Fix: deepseek forward absorb (#5723) · 93c6fb12
  michael-amd authored Apr 25, 2025
```
Co-authored-by: ispobock <ispobaoke@163.com>
```
  93c6fb12
- [PD]: Support Muti Prefill in one node (#5704) · 11e27d09
  IAN authored Apr 26, 2025
```
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  11e27d09
- [PD] Add kvargs table and thread pool for kvcache sender of mooncake (#5738) · 50eda839
  shangmingc authored Apr 25, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  50eda839
- [PD] Better logs (#5715) · c55550cb
  Liangsheng Yin authored Apr 25, 2025
  
  c55550cb
- [Model] Support `ArcticForCausalLM` architecture (Snowflake/snowflake-arctic-instruct) (#5078) · 43fb95c2
  Brayden Zhong authored Apr 25, 2025
```
Co-authored-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
```
  43fb95c2