Commits · 2ed96c7a8a2039191d30a523b0983209c3e91dd1 · change / sglang · GitLab

22 Apr, 2025 4 commits
- fix flashmla bug (#5272) · 2ed96c7a
  lukec authored Apr 23, 2025
  
  2ed96c7a
- [feature] Add H20 fp8_w8a8 FusedMoE config for --n-share-experts-fusion=16 (#5641) · 2aa3f5e2
  saltyfish66 authored Apr 23, 2025
```
Co-authored-by: yuethe <yuethe@tencent.com>
```
  2aa3f5e2
- Remove extra copy in deepseek forward absorb (#5578) · 11b23ae9
  Ke Bao authored Apr 22, 2025
```
Co-authored-by: saienduri <saimanas.enduri@amd.com>
```
  11b23ae9
- Support aiter RMSNorm in AMD (#5510) · 968ef515
  michael-amd authored Apr 21, 2025
```
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
```
  968ef515
21 Apr, 2025 5 commits
- [feature] enable pre compile jit deep_gemm (#5580) · c2942907
  JieXin Liang authored Apr 22, 2025
  
  c2942907
- Add Speculative Decoding Eagle3 topk > 1 (#5318) · 188f0955
  Qingquan Song authored Apr 20, 2025
```
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
```
  188f0955
- [misc] remove is_cuda_available (#5319) · 97cb762b
  JieXin Liang authored Apr 21, 2025
  
  97cb762b
- compressed_tensors: port w8a16 fp8 from vllm (#4852) · 502524e2
  Juwan Yoo authored Apr 21, 2025
  
  502524e2
- check marlin format before attempting conversion (#4675) · 4c764007
  Enrique Shockwave authored Apr 21, 2025
  
  4c764007
20 Apr, 2025 4 commits
- [Feat] upgrade pytorch2.6 (#5417) · 417b44eb
  lukec authored Apr 21, 2025
  
  417b44eb
- Fix DeepEP cannot run on latest master (#5567) · 463d4b74
  fzyzcjy authored Apr 21, 2025
  
  463d4b74
- [perf] introduce deep gemm group_gemm_masked as bmm (#5432) · 99456bca
  JieXin Liang authored Apr 20, 2025
  
  99456bca
- Fix sampler nan check when calling top_k_top_p_sampling_from_probs (#5546) · 20f1c8e3
  Yubo Wang authored Apr 19, 2025
  
  20f1c8e3
19 Apr, 2025 5 commits
- Remove one kernel in per_tensor_quant_mla_fp8 (#5549) · 613b197e
  fzyzcjy authored Apr 20, 2025
  
  613b197e
- simplify the control logic for using shared experts fusion (#5504) · d58e3544
  Xiaoyu Zhang authored Apr 20, 2025
  
  d58e3544
- restruct compressed_tensors_w8a8_fp8 (#5475) · bf86c5e9
  Xiaoyu Zhang authored Apr 19, 2025
  
  bf86c5e9
- Fix DeepGEMM masked cannot be run on groups not being multiple or 4 (#5340) · 1e0806f3
  fzyzcjy authored Apr 19, 2025
  
  1e0806f3
- chore: upgrade sgl-kernel 0.0.9.post2 (#5540) · 2c11f9c2
  Yineng Zhang authored Apr 18, 2025
  
  2c11f9c2
18 Apr, 2025 7 commits
- Revert "Avoid computing lse in Ragged Prefill when there's no prefix.… (#5544) · a6f892e5
  Yineng Zhang authored Apr 18, 2025
  
  a6f892e5
- fix util import (#5542) · 08b518d5
  Yineng Zhang authored Apr 18, 2025
  
  08b518d5
- [Model] Adding Qwen3 and Qwen3MoE (#4693) · 4db463b1
  yhyang201 authored Apr 19, 2025
  
  4db463b1
- Avoid computing lse in Ragged Prefill when there's no prefix. (#5476) · bfa39224
  Wenxuan Tan authored Apr 18, 2025
```
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  bfa39224
- fix bug of VLLM_AVAILABLE not defined (#5497) · e465b08d
  liwenju0 authored Apr 18, 2025
  
  e465b08d
- fix kimi vl running bug after rebase main (#5461) · bed05878
  Xiaoyu Zhang authored Apr 18, 2025
  
  bed05878
- use sglang_per_token_group_quant_fp8 from sgl-kernel instead of trion kernel (#5473) · b2a189dd
  strgrb authored Apr 18, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  b2a189dd
17 Apr, 2025 4 commits
- Enable local attention during decode (#5479) · c776234b
  Chang Su authored Apr 17, 2025
  
  c776234b
- Feat/support encoder model (like bert) (#4887) · 3bface15
  woodx authored Apr 17, 2025
  
  3bface15
- Deprecate disable-mla (#5481) · 4fb05583
  Baizhou Zhang authored Apr 17, 2025
  
  4fb05583
- [Model Support] unsloth/Phi-4-mini bnb model (#4982) · 8f783c19
  eigen authored Apr 16, 2025
```
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  8f783c19
16 Apr, 2025 2 commits
- Clean up imports (#5467) · 177320a5
  Lianmin Zheng authored Apr 16, 2025
  
  177320a5
- Support MHA with chunked prefix cache for DeepSeek chunked prefill (#5113) · a42736bb
  Baizhou Zhang authored Apr 15, 2025
  
  a42736bb
15 Apr, 2025 5 commits
- feat: update model_specific_adjustment (#5344) · fa909dc3
  Yineng Zhang authored Apr 15, 2025
```
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
```
  fa909dc3
- Tiny fix DeepseekScalingRotaryEmbedding always use forward_native (#5406) · 15e91d72
  fzyzcjy authored Apr 15, 2025
  
  15e91d72
- Add H20 dtype fp8_w8a8 shared experts fused MoE kernel tuning configs for DeepSeek V3/R1 (#5291) · 2dd64894
  Ximingwang-09 authored Apr 15, 2025
```
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
```
  2dd64894
- Add A800 shared experts fused MoE kernel tuning configs for DeepSeek V3/R1 (#5368) · 61e7c4dd
  lambert0312 authored Apr 15, 2025
  
  61e7c4dd
- [Fix] Turn off DeepGEMM by default (#5263) · f6772f14
  Baizhou Zhang authored Apr 14, 2025
  
  f6772f14
14 Apr, 2025 4 commits
- apply fused moe gate in ds v3/r1 (#5371) · 38076dea
  Xiaoyu Zhang authored Apr 15, 2025
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  38076dea
- Apply deepseek cuda rope (#5385) · 5e0a9b09
  Ke Bao authored Apr 15, 2025
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  5e0a9b09
- [perf] experimental enhance fp8 per-tensor quant (#5370) · bdde2375
  JieXin Liang authored Apr 15, 2025
  
  bdde2375
- Fix DeepSeek DP Attention + torch compile (#5367) · defede50
  fzyzcjy authored Apr 14, 2025
```
Co-authored-by: ispobock <ispobaoke@163.com>
```
  defede50