Commits · 475e2e378a3cb2de38a40a316c4afba3584c7e9e · change / sglang · GitLab

20 Apr, 2025 16 commits
- [PD] Fix server crash when using batch requests (#5531) · 475e2e37
  fzyzcjy authored Apr 21, 2025
  
  475e2e37
- Tiny improve error message (#5526) · fba86b6b
  fzyzcjy authored Apr 21, 2025
  
  fba86b6b
- Fix torch memory saver not enabled in DP scenario (#5560) · fa2f677e
  fzyzcjy authored Apr 21, 2025
  
  fa2f677e
- Fix DeepEP cannot run on latest master (#5567) · 463d4b74
  fzyzcjy authored Apr 21, 2025
  
  463d4b74
- Fix bench_serving fail when zero warmup requests (#5574) · 9924bbe1
  fzyzcjy authored Apr 21, 2025
  
  9924bbe1
- Release v0.4.5.post2 (#5582) · fbdc94ba
  Lianmin Zheng authored Apr 20, 2025
  
  fbdc94ba
- [Doc]Add instruction for profiling with bench_one_batch (#5581) · b54b5a96
  Baizhou Zhang authored Apr 20, 2025
  
  b54b5a96
- [Fix] fix outlines and xgrammar (#4947) · bca832c7
  JieXin Liang authored Apr 21, 2025
  
  bca832c7
- enable DeepSeek V3 shared_experts_fusion in sm90 (#5571) · d9dd5298
  Xiaoyu Zhang authored Apr 21, 2025
  
  d9dd5298
- Fix BumpAllocator error when no input_ids (#5564) · 0a0dd34e
  fzyzcjy authored Apr 20, 2025
  
  0a0dd34e
- [PD] Fix DeepSeek cannot be run on latest master (#5568) · 80ac527d
  fzyzcjy authored Apr 20, 2025
  
  80ac527d
- [perf] introduce deep gemm group_gemm_masked as bmm (#5432) · 99456bca
  JieXin Liang authored Apr 20, 2025
  
  99456bca
- Fix bench_one_batch producing unnatural results for expert parallel (#5149) · d07e797a
  fzyzcjy authored Apr 20, 2025
  
  d07e797a
- fix hicache write back (#5543) · e2574ee9
  Zhiqiang Xie authored Apr 19, 2025
  
  e2574ee9
- [PD] Support page size > 1 (#5561) · ab4b5606
  Byron Hsu authored Apr 19, 2025
  
  ab4b5606
- Fix sampler nan check when calling top_k_top_p_sampling_from_probs (#5546) · 20f1c8e3
  Yubo Wang authored Apr 19, 2025
  
  20f1c8e3
19 Apr, 2025 9 commits
- Remove one kernel in per_tensor_quant_mla_fp8 (#5549) · 613b197e
  fzyzcjy authored Apr 20, 2025
  
  613b197e
- simplify the control logic for using shared experts fusion (#5504) · d58e3544
  Xiaoyu Zhang authored Apr 20, 2025
  
  d58e3544
- restruct compressed_tensors_w8a8_fp8 (#5475) · bf86c5e9
  Xiaoyu Zhang authored Apr 19, 2025
  
  bf86c5e9
- [PD] Remove the requirement of config file for mooncake backend (#5460) · dca90f1d
  shangmingc authored Apr 19, 2025
  
  dca90f1d
- [PD] Fix no cache connect for recevier (#5534) · 59dd090f
  ybyang authored Apr 19, 2025
  
  59dd090f
- [PD] Tiny fix timeout error when generate (#5545) · 569b032c
  fzyzcjy authored Apr 19, 2025
  
  569b032c
- Make profiler output file names consistent (#5548) · f6a71139
  fzyzcjy authored Apr 19, 2025
  
  f6a71139
- Fix DeepGEMM masked cannot be run on groups not being multiple or 4 (#5340) · 1e0806f3
  fzyzcjy authored Apr 19, 2025
  
  1e0806f3
- chore: upgrade sgl-kernel 0.0.9.post2 (#5540) · 2c11f9c2
  Yineng Zhang authored Apr 18, 2025
  
  2c11f9c2
18 Apr, 2025 11 commits
- Revert "Avoid computing lse in Ragged Prefill when there's no prefix.… (#5544) · a6f892e5
  Yineng Zhang authored Apr 18, 2025
  
  a6f892e5
- fix util import (#5542) · 08b518d5
  Yineng Zhang authored Apr 18, 2025
  
  08b518d5
- [Model] Adding Qwen3 and Qwen3MoE (#4693) · 4db463b1
  yhyang201 authored Apr 19, 2025
  
  4db463b1
- Avoid computing lse in Ragged Prefill when there's no prefix. (#5476) · bfa39224
  Wenxuan Tan authored Apr 18, 2025
```
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  bfa39224
- fix bug of VLLM_AVAILABLE not defined (#5497) · e465b08d
  liwenju0 authored Apr 18, 2025
  
  e465b08d
- fix kimi vl running bug after rebase main (#5461) · bed05878
  Xiaoyu Zhang authored Apr 18, 2025
  
  bed05878
- use sglang_per_token_group_quant_fp8 from sgl-kernel instead of trion kernel (#5473) · b2a189dd
  strgrb authored Apr 18, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  b2a189dd
- Introduce moe_dense_tp_size to fix dense layer errors in DeepSeek V3 + 4x8xH100 (#4836) · 53dcf388
  fzyzcjy authored Apr 18, 2025
  
  53dcf388
- Configuration qwen2_moe.py - qkv_bias now in transformers (#5512) · 1effba4c
  Michael Feil authored Apr 17, 2025
  
  1effba4c
- Fix: fix the exception 'the memory capacity is unbalanced. Some GPUs … (#5426) · 27e9538a
  mlmz authored Apr 18, 2025
```
Co-authored-by: ocss884 <ocss.lin@gmail.com>
```
  27e9538a
- Fix: Incorrect parameters passed to forward_batch_generation (#5506) (#5511) · 211c7b31
  u4lr451 authored Apr 18, 2025
  
  211c7b31
17 Apr, 2025 4 commits
- [Bug fix] use correct func path in deepseek (#5496) · 06d0a3d9
  Xuchun Shang authored Apr 17, 2025
```
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
```
  06d0a3d9
- Refactor DeepSeek decoder layer branches (#5205) · 8beb356f
  fzyzcjy authored Apr 17, 2025
  
  8beb356f
- Enable local attention during decode (#5479) · c776234b
  Chang Su authored Apr 17, 2025
  
  c776234b
- Feat/support encoder model (like bert) (#4887) · 3bface15
  woodx authored Apr 17, 2025
  
  3bface15