Commits · fba86b6b545981ea76fee26f0d454ea08a02b27d · change / sglang · GitLab

20 Apr, 2025 18 commits
- Tiny improve error message (#5526) · fba86b6b
  fzyzcjy authored Apr 21, 2025
  
  fba86b6b
- Add document for LoRA serving (#5521) · 072b4d03
  Baizhou Zhang authored Apr 20, 2025
  
  072b4d03
- Super tiny fix typo (#5559) · 9c434777
  fzyzcjy authored Apr 21, 2025
  
  9c434777
- Fix torch memory saver not enabled in DP scenario (#5560) · fa2f677e
  fzyzcjy authored Apr 21, 2025
  
  fa2f677e
- Fix DeepEP cannot run on latest master (#5567) · 463d4b74
  fzyzcjy authored Apr 21, 2025
  
  463d4b74
- Fix bench_serving fail when zero warmup requests (#5574) · 9924bbe1
  fzyzcjy authored Apr 21, 2025
  
  9924bbe1
- Release v0.4.5.post2 (#5582) · fbdc94ba
  Lianmin Zheng authored Apr 20, 2025
  
  fbdc94ba
- [Doc]Add instruction for profiling with bench_one_batch (#5581) · b54b5a96
  Baizhou Zhang authored Apr 20, 2025
  
  b54b5a96
- [Fix] fix outlines and xgrammar (#4947) · bca832c7
  JieXin Liang authored Apr 21, 2025
  
  bca832c7
- enable DeepSeek V3 shared_experts_fusion in sm90 (#5571) · d9dd5298
  Xiaoyu Zhang authored Apr 21, 2025
  
  d9dd5298
- Fix BumpAllocator error when no input_ids (#5564) · 0a0dd34e
  fzyzcjy authored Apr 20, 2025
  
  0a0dd34e
- [PD] Fix DeepSeek cannot be run on latest master (#5568) · 80ac527d
  fzyzcjy authored Apr 20, 2025
  
  80ac527d
- [perf] introduce deep gemm group_gemm_masked as bmm (#5432) · 99456bca
  JieXin Liang authored Apr 20, 2025
  
  99456bca
- Fix bench_one_batch producing unnatural results for expert parallel (#5149) · d07e797a
  fzyzcjy authored Apr 20, 2025
  
  d07e797a
- Minor update for ROCm variable style (#5562) · c555d794
  Zhaoyi Li authored Apr 20, 2025
  
  c555d794
- fix hicache write back (#5543) · e2574ee9
  Zhiqiang Xie authored Apr 19, 2025
  
  e2574ee9
- [PD] Support page size > 1 (#5561) · ab4b5606
  Byron Hsu authored Apr 19, 2025
  
  ab4b5606
- Fix sampler nan check when calling top_k_top_p_sampling_from_probs (#5546) · 20f1c8e3
  Yubo Wang authored Apr 19, 2025
  
  20f1c8e3
19 Apr, 2025 10 commits
- Remove one kernel in per_tensor_quant_mla_fp8 (#5549) · 613b197e
  fzyzcjy authored Apr 20, 2025
  
  613b197e
- simplify the control logic for using shared experts fusion (#5504) · d58e3544
  Xiaoyu Zhang authored Apr 20, 2025
  
  d58e3544
- restruct compressed_tensors_w8a8_fp8 (#5475) · bf86c5e9
  Xiaoyu Zhang authored Apr 19, 2025
  
  bf86c5e9
- [PD] Remove the requirement of config file for mooncake backend (#5460) · dca90f1d
  shangmingc authored Apr 19, 2025
  
  dca90f1d
- feat: use flashinfer jit package (#5547) · 0961feef
  Yineng Zhang authored Apr 19, 2025
  
  0961feef
- [PD] Fix no cache connect for recevier (#5534) · 59dd090f
  ybyang authored Apr 19, 2025
  
  59dd090f
- [PD] Tiny fix timeout error when generate (#5545) · 569b032c
  fzyzcjy authored Apr 19, 2025
  
  569b032c
- Make profiler output file names consistent (#5548) · f6a71139
  fzyzcjy authored Apr 19, 2025
  
  f6a71139
- Fix DeepGEMM masked cannot be run on groups not being multiple or 4 (#5340) · 1e0806f3
  fzyzcjy authored Apr 19, 2025
  
  1e0806f3
- chore: upgrade sgl-kernel 0.0.9.post2 (#5540) · 2c11f9c2
  Yineng Zhang authored Apr 18, 2025
  
  2c11f9c2
18 Apr, 2025 12 commits
- Revert "Avoid computing lse in Ragged Prefill when there's no prefix.… (#5544) · a6f892e5
  Yineng Zhang authored Apr 18, 2025
  
  a6f892e5
- fix util import (#5542) · 08b518d5
  Yineng Zhang authored Apr 18, 2025
  
  08b518d5
- [Model] Adding Qwen3 and Qwen3MoE (#4693) · 4db463b1
  yhyang201 authored Apr 19, 2025
  
  4db463b1
- Avoid computing lse in Ragged Prefill when there's no prefix. (#5476) · bfa39224
  Wenxuan Tan authored Apr 18, 2025
```
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  bfa39224
- fix bug of VLLM_AVAILABLE not defined (#5497) · e465b08d
  liwenju0 authored Apr 18, 2025
  
  e465b08d
- fix kimi vl running bug after rebase main (#5461) · bed05878
  Xiaoyu Zhang authored Apr 18, 2025
  
  bed05878
- use sglang_per_token_group_quant_fp8 from sgl-kernel instead of trion kernel (#5473) · b2a189dd
  strgrb authored Apr 18, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  b2a189dd
- chore: bump sgl-kernel 0.0.9.post2 (#5518) · f28d8299
  Yineng Zhang authored Apr 17, 2025
  
  f28d8299
- Sgl kernel fused_moe_gate support n_shared_experts (#5440) · 8e09b370
  Xiaoyu Zhang authored Apr 18, 2025
  
  8e09b370
- Introduce moe_dense_tp_size to fix dense layer errors in DeepSeek V3 + 4x8xH100 (#4836) · 53dcf388
  fzyzcjy authored Apr 18, 2025
  
  53dcf388
- Configuration qwen2_moe.py - qkv_bias now in transformers (#5512) · 1effba4c
  Michael Feil authored Apr 17, 2025
  
  1effba4c
- [docs] Fix several consistency issues in sampling_params.md (#5373) · a0fc5bc1
  Michael Yao authored Apr 18, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  a0fc5bc1