Commits · 733446dd36033831496a0fe0905f743b5fc9db7a · change / sglang

12 Aug, 2025 1 commit
- Fuse writing KV buffer into rope kernel (part 1: sgl-kernel) (#9077) · 9aea2555
  fzyzcjy authored Aug 12, 2025
  
  9aea2555
06 Aug, 2025 1 commit
- fix benchmark fp8 blockwise group gemm (#8815) · 1bd53168
  Yuan Luo authored Aug 06, 2025
  
  1bd53168
01 Aug, 2025 2 commits
- fix per token cuda kernel hidden dim cannot divide by 16 (#8543) · db7343c9
  Stefan He authored Aug 01, 2025
  
  db7343c9
- [Kimi K2] dsv3_router_gemm supports NUM_EXPERTS == 384 (#8013) · 6bdd2786
  Peter Pan authored Aug 01, 2025
  
  6bdd2786
31 Jul, 2025 1 commit

update sgl-kernel for EP: kernel part (#8514) · a5f5ab40

Cheng Wan authored Jul 30, 2025


Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>

a5f5ab40

27 Jul, 2025 3 commits
- Use FlashInfer FP4 gemm. (#8241) · 5c9c275b
  Elfie Guo authored Jul 27, 2025
  
  5c9c275b
- Fix bench script making input data on L2 cache (#7739) · e34cf6ad
  fzyzcjy authored Jul 27, 2025
  
  e34cf6ad
- Add nvfp4 scaled mm benchmark. (#8401) · 426b7493
  Qi Yuhang authored Jul 27, 2025
  
  426b7493
25 Jul, 2025 1 commit

[AMD] Add silu_and_mul, gelu_and_mul, gelu_tanh_and_mul, and gelu_quick... · af4b9bae

Hubert Lu authored Jul 24, 2025


[AMD] Add silu_and_mul, gelu_and_mul, gelu_tanh_and_mul, and gelu_quick kernels for AMD GPUs (#7135)
Co-authored-by: yiakwy-xpu-ml-framework-team <961186938@qq.com>
Co-authored-by: HAI <hixiao@gmail.com>

af4b9bae

22 Jul, 2025 1 commit
- [fix] benchmark : routed_scaling_factor is None (#8059) · 0f8b5386
  Peter Pan authored Jul 22, 2025
```
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
```
  0f8b5386
20 Jul, 2025 1 commit
- Add bf16 output option for dsv3_router_gemm kernel (#7999) · 282eb59f
  Baizhou Zhang authored Jul 19, 2025
  
  282eb59f
03 Jul, 2025 2 commits
- [optimize] fuse renormalize into moe_topk_softmax (#7744) · 2998c4bd
  Yi Zhang authored Jul 04, 2025
```
Co-authored-by: ispobock <ispobaoke@gmail.com>
```
  2998c4bd
- Add CUTLASS FP8 Blockscale MoE kernel for Hopper architecture (#7278) · 2c4feaf3
  ayrnb authored Jul 03, 2025
```
Co-authored-by: HydraQYH <QYH820@Outlook.com>
Co-authored-by: TianQiLin666666 <1834987979@qq.com>
```
  2c4feaf3
30 Jun, 2025 1 commit
- Add dsv3 router gemm kernel (#7627) · 7248272c
  Baizhou Zhang authored Jun 29, 2025
  
  7248272c
29 Jun, 2025 1 commit
- Add dsv3 fused a gemm to sgl-kernel (#7630) · 04b35190
  Ke Bao authored Jun 29, 2025
  
  04b35190
25 Jun, 2025 1 commit
- Fuse sorted_token_ids padding to moe_align_block_size kernel (#7437) · 57ab7769
  Ke Bao authored Jun 25, 2025
  
  57ab7769
23 Jun, 2025 1 commit

feat: integrate deepgemm into EPMoE (#6821) · 506c4928

xutizhou authored Jun 23, 2025


Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: TianQiLin666666 <1834987979@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>

506c4928

14 Jun, 2025 1 commit
- [fix] fix cutlass_mla_backend with cuda_graph and add sm_scale for sgl-kernel cutlass_mla (#7184) · ab1a4fa5
  JieXin Liang authored Jun 15, 2025
  
  ab1a4fa5
13 Jun, 2025 1 commit
- Remove 200us slow concat kernel (part 1: kernel) (#7145) · aa46ed34
  fzyzcjy authored Jun 13, 2025
  
  aa46ed34
12 Jun, 2025 1 commit
- [sgl-kernel] Add cuda kernel for moe_ep_silu_and_mul (#6919) · 84727a51
  Yuan Luo authored Jun 12, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
```
  84727a51
09 Jun, 2025 1 commit
- [perf][sgl-kernel] extend cutlass_mla_decode to support num_head < 128 (#6929) · 18efb5e8
  JieXin Liang authored Jun 09, 2025
  
  18efb5e8
05 Jun, 2025 1 commit
- [EP] Add cuda kernel for moe_ep_post_reorder (#6837) · 43baba64
  Yuan Luo authored Jun 05, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
```
  43baba64
04 Jun, 2025 1 commit
- fix ep_moe_reorder kernel bugs (#6858) · bd75690f
  Xiaoyu Zhang authored Jun 04, 2025
```
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
```
  bd75690f
02 Jun, 2025 1 commit
- [EP] Add cuda kernel for moe_ep_pre_reorder (#6699) · 55444ed6
  Yuan Luo authored Jun 02, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
```
  55444ed6
29 May, 2025 1 commit
- refactor apply_w8a8_block_fp8_linear in fp (#6545) · 485a023b
  ChangyiYang authored May 29, 2025
  
  485a023b
22 May, 2025 1 commit

[1/2] Support Qserve (#6457) · 4d643f6c

HandH1998 authored May 22, 2025


Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>

4d643f6c

12 May, 2025 2 commits
- Revert "fix some typos" (#6244) · e8e18dcd
  Lianmin Zheng authored May 12, 2025
  
  e8e18dcd
- fix some typos (#6209) · d738ab52
  applesaucethebun authored May 12, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  d738ab52
11 May, 2025 1 commit
- Add typo checker in pre-commit (#6179) · 2ce87935
  applesaucethebun authored May 11, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  2ce87935
12 Apr, 2025 1 commit
- update variable naming and comments for rocm (#5299) · 3c9740d2
  Zhaoyi Li authored Apr 12, 2025
  
  3c9740d2
11 Apr, 2025 1 commit
- fix: remove cublas_grouped_gemm (#5307) · 136b8e6a
  Yineng Zhang authored Apr 11, 2025
  
  136b8e6a
10 Apr, 2025 1 commit
- reduce moe_align_block_size_kernel small batch mode overhead (#5086) · f730362e
  Xiaoyu Zhang authored Apr 10, 2025
  
  f730362e
09 Apr, 2025 1 commit
- sgl-kernel use cutlass latest version for fp8 blockwise gemm (#5207) · ebf495f0
  Yi Zhang authored Apr 10, 2025
  
  ebf495f0
03 Apr, 2025 1 commit
- [sgl-kernel] per token group quant support COLUMN MAJOR (#4817) · 2c8fd993
  Xiaoyu Zhang authored Apr 03, 2025
  
  2c8fd993
29 Mar, 2025 1 commit
- Add deepseek style fused moe group gate selection kernel (#4530) · 45dcfc2e
  Qingquan Song authored Mar 29, 2025
  
  45dcfc2e
24 Mar, 2025 1 commit
- [Quant Kernel] refactored per token group quant fp8 to support int8 up-to 2x faster (#4396) · 65c24c28
  Chunan Zeng authored Mar 23, 2025
  
  65c24c28
16 Mar, 2025 1 commit
- [Fix] use `torch.cat` instead of `torch.concat` to prevent entering the... · 1a3fa75f
  JieXin Liang authored Mar 16, 2025
```
[Fix] use `torch.cat` instead of `torch.concat` to prevent entering the `Autograd` backends. (#4466)
```
  1a3fa75f
14 Mar, 2025 1 commit
- Add moe topk softmax templated from vllm (#4302) · 61e4433c
  Qingquan Song authored Mar 14, 2025
  
  61e4433c
13 Mar, 2025 2 commits
- fix accuracy issue (#4376) · 2937387a
  Yineng Zhang authored Mar 13, 2025
  
  2937387a
- Fix per token fp8 quant precision (#4362) · 4068e012
  Qingquan Song authored Mar 12, 2025
  
  4068e012