Commits · f58b929a5185029c88cea378901d9e011380f173 · change / sglang · GitLab

13 Apr, 2025 3 commits
- chore: upgrade sgl-kernel 0.0.8.post3 (#5342) · f58b929a
  Yineng Zhang authored Apr 13, 2025
  
  f58b929a
- Fix: Ensure tensors for dist.broadcast match NCCL backend device (#5322) · 8311b07f
  mlmz authored Apr 13, 2025
  
  8311b07f
- fix: determine if flashinfer is installed (#5336) · 7d3b7c87
  Yineng Zhang authored Apr 12, 2025
  
  7d3b7c87
12 Apr, 2025 8 commits
- Support server based rollout in Verlengine (#4848) · bc92107b
  tianlian yi authored Apr 13, 2025
```
Co-authored-by: Jin Pan <jpan236@wisc.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Jinn <47354855+jhinpan@users.noreply.github.com>
```
  bc92107b
- refine fused_moe tuning docs (#5294) · 3e4794aa
  Xiaoyu Zhang authored Apr 13, 2025
  
  3e4794aa
- Delete python/sglang/srt/layers/moe/fused_moe_triton/configs/E=257,N=… (#5321) · 690ec205
  Xiaoyu Zhang authored Apr 13, 2025
  
  690ec205
- feat: use fa3 mla by default on hopper (#5210) · 57de7c6b
  Yineng Zhang authored Apr 12, 2025
```
Co-authored-by: yundai424 <yundai424@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
```
  57de7c6b
- Fix fa3 window size setup (#5316) · aea98512
  Qingquan Song authored Apr 11, 2025
  
  aea98512
- Fix w8a8_int8 model shared experts fusion load weights error (#5120) · 1b1b47a9
  lambert0312 authored Apr 12, 2025
  
  1b1b47a9
- [VLM] Adopt fast image processor by default (#5065) · 34ef6c81
  Mick authored Apr 12, 2025
  
  34ef6c81
- fix: use deepgemm only on hopper (#5310) · 61172091
  Yineng Zhang authored Apr 11, 2025
  
  61172091
11 Apr, 2025 9 commits
- feat: add blackwell Dockerfile (#5302) · f774a0d2
  Yineng Zhang authored Apr 11, 2025
  
  f774a0d2
- remove moe_align_block_size torch.zeros in small batch/expert mode (#5298) · 60bcbf2a
  Xiaoyu Zhang authored Apr 12, 2025
  
  60bcbf2a
- [PD] fix: skip warmup request in disaggregation mode to prevent crash on timeout (#5292) · c35dcfdb
  Yusong Gao authored Apr 11, 2025
  
  c35dcfdb
- [fix] fix mrope positions not picked up (#5265) · e53a0b3d
  Mick authored Apr 11, 2025
  
  e53a0b3d
- Support `--enable-llama4-multimodal` (#5254) · 038bc5d5
  Cheng Wan authored Apr 11, 2025
  
  038bc5d5
- Optimize GPU memory usage in FlashAttentionBackend's strided indexing (#5262) · aee62d74
  Chang Su authored Apr 11, 2025
```
Co-authored-by: ch-wan <cwan39@gatech.edu>
```
  aee62d74
- Optimize attention in llama4 (#5127) · cd7e32e2
  fzyzcjy authored Apr 11, 2025
  
  cd7e32e2
- ROCm/AITER CK_MoE: update 2-stage kernels & support both Activations (#5228) · 88799448
  HAI authored Apr 10, 2025
  
  88799448
- Fix torch.compile cacheing (#5259) · a879811c
  Richard Zou authored Apr 10, 2025
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  a879811c
10 Apr, 2025 6 commits
- Update deps for mllama4 (#5215) · 1078396f
  Ke Bao authored Apr 11, 2025
  
  1078396f
- [PD] Add get_contiguous_buf_infos interface for MLATokenToKVPool (#5204) · 7e4f72dd
  Teng Ma authored Apr 10, 2025
  
  7e4f72dd
- [PD] Support KV transfer with mooncake (#4880) · 4c31ae9f
  Teng Ma authored Apr 10, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Co-authored-by: shangmingc <csmthu@gmail.com>
```
  4c31ae9f
- reduce moe_align_block_size_kernel small batch mode overhead (#5086) · f730362e
  Xiaoyu Zhang authored Apr 10, 2025
  
  f730362e
- Fix DeepSeek error when using DeepEP mode (#5190) · e3c4bd31
  fzyzcjy authored Apr 10, 2025
  
  e3c4bd31
- [metrics] Add in queue metrics (#4444) · 5db37c86
  Stefan He authored Apr 09, 2025
  
  5db37c86
09 Apr, 2025 14 commits
- fix: log warning when disable cuda graph (#5209) · 4cb53ecd
  Yineng Zhang authored Apr 09, 2025
  
  4cb53ecd
- Add H20 dtype fp8_w8a8 fused MoE kernel tuning configs for DeepSeek V3/R1 (#5196) · 456b008b
  Zhaoyang Hao authored Apr 10, 2025
  
  456b008b
- update grok test (#5171) · 7f875f12
  saienduri authored Apr 09, 2025
  
  7f875f12
- model: support mllama4 (#5144) · fbebcb7a
  Mick authored Apr 10, 2025
  
  fbebcb7a
- Support Llama4 fp8 inference (#5194) · 40652482
  HandH1998 authored Apr 09, 2025
```
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: zhyncs <me@zhyncs.com>
```
  40652482
- Optimize topk operation in llama4 (#5128) · 86a876d8
  fzyzcjy authored Apr 09, 2025
  
  86a876d8
- Fix ci test "test_eval_fp8_accuracy" failed (#5185) · 92823069
  kk authored Apr 09, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
```
  92823069
- Let `bench_one_batch` support `enable_dp_attention` (#4058) · 61970b08
  fzyzcjy authored Apr 09, 2025
  
  61970b08
- [DeepEP] fix: import buffer error (#5179) · 76c48a09
  Cheng Wan authored Apr 09, 2025
  
  76c48a09
- fix: use DeepEPDispatcher on CUDA (#5180) · 90caf06c
  Yineng Zhang authored Apr 08, 2025
  
  90caf06c
- feat: add DeepGEMM build warning (#5176) · 6669d127
  Yineng Zhang authored Apr 08, 2025
```
Co-authored-by: grimoire <streetyao@live.com>
```
  6669d127
- [Fix] DeepEP Compatibility with Low Latency (#5068) · bc3f6db2
  Jinyan Chen authored Apr 09, 2025
```
Co-authored-by: ch-wan <cwan39@gatech.edu>
```
  bc3f6db2
- [Bugfix] Fix index out of bounds in local attention with large sequences (#5173) · aac531c5
  Chang Su authored Apr 08, 2025
  
  aac531c5
- Fix multimodal hashing error (#5174) · 466899e6
  fzyzcjy authored Apr 09, 2025
  
  466899e6