Commits · 76f44c2a8d576fd2fea7fc7c2f25030bcb98374a · change / sglang

10 Apr, 2025 2 commits
- Fix deepseek-v3 with torch.compile in PyTorch 2.6. (#5213) · 76f44c2a
  Richard Zou authored Apr 10, 2025
  
  76f44c2a
- reduce moe_align_block_size_kernel small batch mode overhead (#5086) · f730362e
  Xiaoyu Zhang authored Apr 10, 2025
  
  f730362e
09 Apr, 2025 1 commit
- sgl-kernel use cutlass latest version for fp8 blockwise gemm (#5207) · ebf495f0
  Yi Zhang authored Apr 10, 2025
  
  ebf495f0
08 Apr, 2025 1 commit

Add optimized native kernels in sgl-kernel (#5150) · a73c4df4

Ma Mingfei authored Apr 09, 2025


Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>
Co-authored-by: blzheng <beilei.zheng@intel.com>

a73c4df4

05 Apr, 2025 1 commit
- sgl-kernel transfer custom allreduce from trt kernel to vllm kernel (#5079) · bcbbf519
  Yi Zhang authored Apr 06, 2025
  
  bcbbf519
03 Apr, 2025 2 commits
- [Fix] fix fa3 build at cu118 (#5036) · b8b6008f
  yinfan98 authored Apr 04, 2025
  
  b8b6008f
- [sgl-kernel] per token group quant support COLUMN MAJOR (#4817) · 2c8fd993
  Xiaoyu Zhang authored Apr 03, 2025
  
  2c8fd993
31 Mar, 2025 2 commits
- [Build] Fix cuda12.8 build error in nvfp4_scaled_mm_kernels.cu (#4953) · ee47a6c1
  Yuhong Guo authored Apr 01, 2025
  
  ee47a6c1
- [Fix] revert clean m.def for cudagraph (#4944) · c7457191
  yinfan98 authored Mar 31, 2025
  
  c7457191
30 Mar, 2025 3 commits
- [feat] add fa3 in sgl-kernel (#4902) · 37c66ec8
  yinfan98 authored Mar 31, 2025
```
Co-authored-by: Sleepcoo <Sleepcoo@gmail.com>
```
  37c66ec8
- fix bmm fp8 (#4926) · 195a09f5
  Yineng Zhang authored Mar 30, 2025
  
  195a09f5
- [Misc] Clean m.def and add Development Tips (#4890) · 0d7fe866
  yinfan98 authored Mar 30, 2025
  
  0d7fe866
29 Mar, 2025 2 commits
- Add deepseek style fused moe group gate selection kernel (#4530) · 45dcfc2e
  Qingquan Song authored Mar 29, 2025
  
  45dcfc2e
- fix sgl-kernel cu118 build (#4872) · ec3ee028
  Yineng Zhang authored Mar 28, 2025
  
  ec3ee028
27 Mar, 2025 1 commit

support cmake for sgl-kernel (#4706) · 8bf6d7f4

Yineng Zhang authored Mar 27, 2025


Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: yinfan98 <1106310035@qq.com>

8bf6d7f4

26 Mar, 2025 1 commit
- Fix shared memory OOM on sm86 GPUs. (#4797) · 45fdf1f7
  Yi Pan authored Mar 27, 2025
  
  45fdf1f7
25 Mar, 2025 1 commit
- Support FP4 gemm (1/2) (#3899) · e9f8e423
  Trevor Morris authored Mar 24, 2025
  
  e9f8e423
24 Mar, 2025 2 commits
- [Quant Kernel] refactored per token group quant fp8 to support int8 up-to 2x faster (#4396) · 65c24c28
  Chunan Zeng authored Mar 23, 2025
  
  65c24c28
- [ROCm] Enable MTP (NextN) on AMD GPU (#4631) · af6535e7
  Alex Sun authored Mar 24, 2025
  
  af6535e7
22 Mar, 2025 2 commits
- [1/3] fix dsv3 awq issue (#4556) · 321ab756
  AniZpZ authored Mar 22, 2025
```
Co-authored-by: leoneo <1320612015@qq.com>
```
  321ab756
- Speed up per token and per tensor quant by 15% (#4639) · 6a384d5c
  Chunan Zeng authored Mar 22, 2025
  
  6a384d5c
20 Mar, 2025 1 commit
- Support fp8 gemm for blackwell (#4558) · ad4e58bf
  Shu Wang authored Mar 20, 2025
  
  ad4e58bf
17 Mar, 2025 2 commits
- Support serving DeepSeek-R1-Channel-INT8 with 32 L40S. (#4418) · 75b65648
  Wenbo Yang authored Mar 17, 2025
  
  75b65648
- [ROCm] enable moe topk softmax in amd (#4448) · 9b8333d9
  yiakwy-xpu-ml-framework-team authored Mar 17, 2025
  
  9b8333d9
16 Mar, 2025 3 commits
- fix custom allreduce performance/accuracy problem (#4477) · 25e1816e
  Yi Zhang authored Mar 17, 2025
  
  25e1816e
- [Eagle] Remove the greedy branch and some redundant code (#4363) · 1b859295
  Ying Sheng authored Mar 16, 2025
```
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  1b859295
- Add greedy verification kernel (#4383) · 52a34d74
  Ying Sheng authored Mar 16, 2025
  
  52a34d74
14 Mar, 2025 1 commit
- Add moe topk softmax templated from vllm (#4302) · 61e4433c
  Qingquan Song authored Mar 14, 2025
  
  61e4433c
13 Mar, 2025 3 commits
- fix accuracy issue (#4376) · 2937387a
  Yineng Zhang authored Mar 13, 2025
  
  2937387a
- Fix per token fp8 quant precision (#4362) · 4068e012
  Qingquan Song authored Mar 12, 2025
  
  4068e012
- feat: support ep size < 32 for sgl kernel (#4348) · 817d4370
  Shi Shuai authored Mar 13, 2025
  
  817d4370
12 Mar, 2025 3 commits
- Support Blackwell Block Scale FP8 Gemm (#4278) · 7c866711
  Elfie Guo authored Mar 12, 2025
  
  7c866711
- Add awq dequantize kernel to sgl with 1x to 3x speedup (#4104) · 07f94463
  Rex authored Mar 12, 2025
  
  07f94463
- Remove vllm ops scaled fp8 quant and accelerate per token quant by 20-28% (#4215) · e0917e6b
  Stefan He authored Mar 12, 2025
```
Co-authored-by: Stefan He <bhe@linkedin.com>
```
  e0917e6b
11 Mar, 2025 1 commit
- [AMD] Fix rocm sgl-kernel missing modules error (#4311) · 690e1f23
  yigex authored Mar 12, 2025
```
Co-authored-by: yiakwy-xpu-ml-framework-team <leiwang2@amd.com>
```
  690e1f23
10 Mar, 2025 5 commits
- Optimize rope in sgl kernel (#4267) · cf0ccd40
  Lianmin Zheng authored Mar 10, 2025
  
  cf0ccd40
- fix per_token_group_quant_fp8 illegal memory when num_groups % 16 != 0 (#4231) · 23308a90
  Xiaoyu Zhang authored Mar 10, 2025
  
  23308a90
- Simplify tests & Fix trtllm custom allreduce registration (#4252) · aa957102
  Lianmin Zheng authored Mar 10, 2025
  
  aa957102
- Move activation.cu to sgl-kernel/elementwise (#4250) · 7c0541b3
  Lianmin Zheng authored Mar 09, 2025
  
  7c0541b3
- Minor style fix for sgl-kernel (#4243) · 730d084f
  Lianmin Zheng authored Mar 09, 2025
  
  730d084f