Commits · 8cd42504011c481faf8eeb047a1b9ccc9523b4b9 · change / sglang · GitLab

22 Mar, 2025 1 commit
- [quantization] fix channelwise conversion with scalar weight scale (#4596) · 8cd42504
  Yun Dai authored Mar 22, 2025
  
  8cd42504
21 Mar, 2025 2 commits
- Set deepgemm to the default value in the hopper architecture. (#4613) · 4c56e5db
  lukec authored Mar 21, 2025
  
  4c56e5db
- fix SUPPORT_CUTLASS_BLOCK_FP8 flag (#4640) · 7b5fc719
  Cheng Wan authored Mar 21, 2025
  
  7b5fc719
19 Mar, 2025 1 commit
- Create col-major and tma-aligned x_scale for deep_gemm.gemm_fp8_fp8_bf16_nt (#4515) · f9c53cbb
  strgrb authored Mar 19, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  f9c53cbb
18 Mar, 2025 2 commits
- cleanup deps 3/n (#4541) · c16b33cc
  Yineng Zhang authored Mar 18, 2025
  
  c16b33cc
- [Hotfix] solve fp8 w8a8 ci test fail (#4531) · dd865bef
  Xiaoyu Zhang authored Mar 18, 2025
  
  dd865bef
17 Mar, 2025 3 commits
- sglang quant module remove vllm dependency (#4507) · 9b81f9bd
  Xiaoyu Zhang authored Mar 18, 2025
  
  9b81f9bd
- [ROCm] fix dtype (#4510) · 5f9b2c62
  yiakwy-xpu-ml-framework-team authored Mar 17, 2025
  
  5f9b2c62
- Support Online Quantization for W8A8 (#4485) · ef3c2dd0
  Stefan He authored Mar 17, 2025
  
  ef3c2dd0
13 Mar, 2025 4 commits
- Revert "[XPU][CPU] Enable the native path of DeepSeek" (#4367) · 45de8971
  Lianmin Zheng authored Mar 12, 2025
  
  45de8971
- [XPU][CPU] Enable the native path of DeepSeek (#4086) · 71046fcd
  Meng, Hengyu authored Mar 13, 2025
```
Co-authored-by: Zhang, Liangang <liangang.zhang@intel.com>
```
  71046fcd
- Support page size > 1 (#4356) · c76040e3
  Lianmin Zheng authored Mar 12, 2025
  
  c76040e3
- [FIX] fix incorrect output when enable both deepgemm and torch compile (#4359) · 85ef7f64
  AniZpZ authored Mar 13, 2025
```
Co-authored-by: xuyongfei.xyf <xuyongfei.xyf@antgroup.com>
```
  85ef7f64
12 Mar, 2025 1 commit
- unify is_cuda and is_hip (#4321) · d1da58e2
  Yineng Zhang authored Mar 11, 2025
  
  d1da58e2
11 Mar, 2025 3 commits
- Add H20 tuning configs support DeepSeek V3/R1 INT8(block-wise) (#4220) · 0f2a2e3c
  Ximingwang-09 authored Mar 12, 2025
```
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
```
  0f2a2e3c
- linear support deepgemm (#4199) · dce303e2
  lukec authored Mar 11, 2025
```
Co-authored-by: yinfan98 <1106310035@qq.com>
```
  dce303e2
- Add A800 tuning configs support DeepSeek V3/R1 BF16 and INT8(block-wise) (#4136) · d3ecd632
  lambert0312 authored Mar 11, 2025
  
  d3ecd632
10 Mar, 2025 3 commits
- Amd test fp8 (#4261) · 2ac189ed
  HandH1998 authored Mar 11, 2025
  
  2ac189ed
- Fix quantization and nightly tests (#4258) · 00d25a7f
  Lianmin Zheng authored Mar 10, 2025
  
  00d25a7f
- Clean up fp8 support (#4230) · e8a69e4d
  Lianmin Zheng authored Mar 09, 2025
  
  e8a69e4d
09 Mar, 2025 1 commit
- Apply sgl w8a8 fp8 kernel (#3148) · 0dd6cda2
  HandH1998 authored Mar 09, 2025
  
  0dd6cda2
07 Mar, 2025 1 commit

[Feature] DeepSeek V3/R1 INT8 Quantization (channel-wise) (#3888) · c7f25446

HandH1998 authored Mar 07, 2025


Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: b0urnee <2769086541@qq.com>

c7f25446

06 Mar, 2025 1 commit
- ROCm: enable trillion-parameter MoE models with INT4-FP8 single node (#4152) · 13bc39c5
  HAI authored Mar 06, 2025
  
  13bc39c5
05 Mar, 2025 2 commits
- ROCM: AITER BLOCK GEMM (#4075) · 5be8f1ed
  yigex authored Mar 05, 2025
  
  5be8f1ed
- [QUANT] Add GPTQModel Dynamic Quantization + `lm_head` Quantization (#3790) · 56a724eb
  Qubitium-ModelCloud authored Mar 05, 2025
```
Signed-off-by: ZX-ModelCloud <zx@modelcloud.ai>
Co-authored-by: ZX-ModelCloud <zx@modelcloud.ai>
```
  56a724eb
04 Mar, 2025 2 commits
- ROCm: update aiter and its usage to fused moe (bloat16, fp8, fp8 block-quant) (#4053) · 51d25405
  HAI authored Mar 04, 2025
  
  51d25405
- Fix: #3988 using blockwise_int8 (#4023) · 12f2e6c3
  Xihuai Wang authored Mar 04, 2025
  
  12f2e6c3
03 Mar, 2025 2 commits

Misc clean up; Remove the support of jump forward (#4032) · 935cda94
Lianmin Zheng authored Mar 03, 2025

935cda94

Support penalty in overlap mode; return logprob with chunked prefill; improve... · ac238727

Lianmin Zheng authored Mar 03, 2025


Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

ac238727

27 Feb, 2025 1 commit
- Tuning Script for Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3922) · b0df5d24
  laixin authored Feb 27, 2025
```
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
```
  b0df5d24
24 Feb, 2025 1 commit
- Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730) · 1a6e9757
  laixin authored Feb 24, 2025
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  1a6e9757
21 Feb, 2025 2 commits
- Add support for nvidia modelopt fp8 kv cache (#3223) · c66b2c9c
  Zhiyu authored Feb 21, 2025
  
  c66b2c9c
- AMD/ROCm: update AITER repo to ROCm/aiter (#3747) · 5c54ef03
  HAI authored Feb 21, 2025
  
  5c54ef03
20 Feb, 2025 1 commit
- add control for cutlass fp8 blockwise gemm (#3727) · 1eb8eade
  yizhang2077 authored Feb 20, 2025
  
  1eb8eade
17 Feb, 2025 3 commits
- [ROCm] Add additional block quant GEMM tuning configs for AMD GPUs. (#3616) · 2eab1132
  Wen-Heng (Jack) Chung authored Feb 17, 2025
```
Co-authored-by: HAI <hixiao@gmail.com>
```
  2eab1132
- support multi-gpu block-gemm tuning (#3639) · c38f3aed
  Xiaoyu Zhang authored Feb 18, 2025
  
  c38f3aed
- Revert "[ROCm] Use `tl.range()` in block GEMM kernels with `num_stage… (#3632) · 5f1a485d
  Yineng Zhang authored Feb 17, 2025
  
  5f1a485d
16 Feb, 2025 1 commit
- [ROCm] Use `tl.range()` in block GEMM kernels with `num_stages` set by host. (#3535) · 03caefeb
  Wen-Heng (Jack) Chung authored Feb 16, 2025
```
Co-authored-by: HAI <hixiao@gmail.com>
```
  03caefeb
13 Feb, 2025 1 commit
- [ROCm] Add ROCm tuning configs for AMD Instinct MI325X. (#3536) · 871a4aa1
  Wen-Heng (Jack) Chung authored Feb 12, 2025
  
  871a4aa1
12 Feb, 2025 1 commit
- integrate blockwise fp8 kernel (#3529) · 98eecbda
  yizhang2077 authored Feb 13, 2025
  
  98eecbda