Commits · 400ad6601933484ecfb00a57e7ac5c72d79fab43 · change / sglang · GitLab

25 Mar, 2025 1 commit
- Support FP4 gemm (1/2) (#3899) · e9f8e423
  Trevor Morris authored Mar 24, 2025
  
  e9f8e423
24 Mar, 2025 1 commit
- [Quant Kernel] refactored per token group quant fp8 to support int8 up-to 2x faster (#4396) · 65c24c28
  Chunan Zeng authored Mar 23, 2025
  
  65c24c28
22 Mar, 2025 1 commit
- [1/3] fix dsv3 awq issue (#4556) · 321ab756
  AniZpZ authored Mar 22, 2025
```
Co-authored-by: leoneo <1320612015@qq.com>
```
  321ab756
17 Mar, 2025 1 commit
- Support serving DeepSeek-R1-Channel-INT8 with 32 L40S. (#4418) · 75b65648
  Wenbo Yang authored Mar 17, 2025
  
  75b65648
16 Mar, 2025 4 commits
- Revert "feat: Add FlashMLA submodule (#4449)" (#4470) · 9971dc22
  Yineng Zhang authored Mar 16, 2025
  
  9971dc22
- Add greedy verification kernel (#4383) · 52a34d74
  Ying Sheng authored Mar 16, 2025
  
  52a34d74
- [Fix] use `torch.cat` instead of `torch.concat` to prevent entering the... · 1a3fa75f
  JieXin Liang authored Mar 16, 2025
```
[Fix] use `torch.cat` instead of `torch.concat` to prevent entering the `Autograd` backends. (#4466)
```
  1a3fa75f
- feat: Add FlashMLA submodule (#4449) · 81f431ed
  Shi Shuai authored Mar 16, 2025
```
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
```
  81f431ed
14 Mar, 2025 1 commit
- Add moe topk softmax templated from vllm (#4302) · 61e4433c
  Qingquan Song authored Mar 14, 2025
  
  61e4433c
13 Mar, 2025 2 commits
- fix accuracy issue (#4376) · 2937387a
  Yineng Zhang authored Mar 13, 2025
  
  2937387a
- Fix per token fp8 quant precision (#4362) · 4068e012
  Qingquan Song authored Mar 12, 2025
  
  4068e012
12 Mar, 2025 1 commit
- Add awq dequantize kernel to sgl with 1x to 3x speedup (#4104) · 07f94463
  Rex authored Mar 12, 2025
  
  07f94463
10 Mar, 2025 3 commits
- fix per_token_group_quant_fp8 illegal memory when num_groups % 16 != 0 (#4231) · 23308a90
  Xiaoyu Zhang authored Mar 10, 2025
  
  23308a90
- Simplify tests & Fix trtllm custom allreduce registration (#4252) · aa957102
  Lianmin Zheng authored Mar 10, 2025
  
  aa957102
- DeepGemm integrate to sgl-kernel (#4165) · c553e160
  laixin authored Mar 10, 2025
```
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: HandH1998 <1335248067@qq.com>
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
Co-authored-by: yinfan98 <1106310035@qq.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  c553e160
09 Mar, 2025 1 commit
- Rename files in sgl kernel to avoid nested folder structure (#4213) · 8abf74e3
  Lianmin Zheng authored Mar 08, 2025
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  8abf74e3
08 Mar, 2025 1 commit
- Remove the vllm dependency from the moe_align function (#4164) · b93ef5e5
  lukec authored Mar 08, 2025
```
Co-authored-by: Hongbosherlock <hongbosherlock@gmail.com>
```
  b93ef5e5
07 Mar, 2025 2 commits
- Add sgl_per_token_quant_fp8 (#4089) · 63ee26d1
  Stefan He authored Mar 06, 2025
  
  63ee26d1
- [quant kernel] sgl-kernel support per_tensor_quant fp8 (#3786) · ad55f171
  Xiaoyu Zhang authored Mar 07, 2025
  
  ad55f171
03 Mar, 2025 2 commits
- Reorganize python source files in sgl-kernel with multiple files (#4027) · 110e0066
  Lianmin Zheng authored Mar 03, 2025
  
  110e0066
- Reorganize c++ source files in sgl-kernel with multiple folders (#4025) · 6b45a21d
  Lianmin Zheng authored Mar 03, 2025
  
  6b45a21d
01 Mar, 2025 1 commit
- Revert "[MOE] enable efficient moe_alignment multi-blocks execution (3x~6x)" (#3982) · 18bb216c
  Chayenne authored Feb 28, 2025
  
  18bb216c
28 Feb, 2025 1 commit
- [MOE] enable efficient moe_alignment multi-blocks execution (3x~6x) (#3613) · 1c96fa86
  yiakwy-xpu-ml-framework-team authored Feb 28, 2025
  
  1c96fa86
18 Feb, 2025 1 commit
- [Feature] Apply Cublas Grouped Gemm kernel (#3629) · 67fc595b
  Baizhou Zhang authored Feb 17, 2025
  
  67fc595b
12 Feb, 2025 1 commit
- support blockwise fp8 matmul kernel (#3267) · 640363ad
  yizhang2077 authored Feb 13, 2025
  
  640363ad
11 Feb, 2025 1 commit
- optimize per token group quant fp8 (#3490) · bb418ced
  Xiaoyu Zhang authored Feb 11, 2025
  
  bb418ced
07 Feb, 2025 1 commit
- support speculative decoding kernel in sgl-kernel (#3373) · f9905d59
  Yineng Zhang authored Feb 07, 2025
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  f9905d59
06 Feb, 2025 1 commit
- clean moe align block kernel code and add acc test (#3332) · ad349985
  Xiaoyu Zhang authored Feb 06, 2025
  
  ad349985
27 Jan, 2025 3 commits
- cleanup sgl-kernel kernels (#3175) · 827aa873
  Yineng Zhang authored Jan 27, 2025
  
  827aa873
- [kernel] Fix position ids in rope (#3173) · 514f37c3
  Byron Hsu authored Jan 27, 2025
  
  514f37c3
- [kernel] Integrate flashinfer's rope with higher precision and better perf (#3134) · fb11a439
  Byron Hsu authored Jan 26, 2025
  
  fb11a439
26 Jan, 2025 2 commits
- support w8a8 fp8 kernel with CUTLASS (#3047) · 82392da8
  HandH1998 authored Jan 26, 2025
```
Co-authored-by: yych0745 <1398089567@qq.com>
```
  82392da8
- minor: cleanup sgl-kernel (#3143) · 95f789ad
  Yineng Zhang authored Jan 26, 2025
  
  95f789ad
25 Jan, 2025 1 commit
- support fp32 in sampling_scaling_penalties kernel (#3121) · 5d9d15e7
  Xiaoyu Zhang authored Jan 25, 2025
  
  5d9d15e7
23 Jan, 2025 5 commits
- feat: integrate sampling kernels into sgl-kernel (#3086) · 5de4051b
  Yineng Zhang authored Jan 24, 2025
```
Co-authored-by: Zihao Ye <expye@outlook.com>
```
  5de4051b
- [hotfix] fix test_sampling_scaling_penalties.py ci test (#3084) · e0cd65c2
  Xiaoyu Zhang authored Jan 24, 2025
  
  e0cd65c2
- use flashinfer vec_dtypes in sgl_kernel (#3083) · f1b68618
  Xiaoyu Zhang authored Jan 23, 2025
  
  f1b68618
- sync flashinfer and update sgl-kernel tests (#3081) · 0da0989a
  Yineng Zhang authored Jan 23, 2025
  
  0da0989a
- support lightning_attention_decode in sgl-kernel for MiniMax-Text-01 (#3030) · ac2dc35d
  Xiaoyu Zhang authored Jan 23, 2025
  
  ac2dc35d
22 Jan, 2025 1 commit
- feat: integrate bmm_fp8 kernel into sgl-kernel (#3056) · bf669606
  Yineng Zhang authored Jan 23, 2025
  
  bf669606