Commits · 60508ffda91c22e4cde3b18f149d222211db8886 · OpenDAS / vllm_cscc

18 Dec, 2024 1 commit

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd

18 Nov, 2024 1 commit
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
22 Oct, 2024 1 commit
- [Bugfix] Fix spurious "No compiled cutlass_scaled_mm ..." for W8A8 on Turing (#9487) · d1e82408
  Lucas Wilkinson authored Oct 22, 2024
  
  d1e82408
04 Oct, 2024 1 commit
- [CI/Build] Per file CUDA Archs (improve wheel size and dev build times) (#8845) · aeb37c2a
  Lucas Wilkinson authored Oct 03, 2024
  
  aeb37c2a
06 Aug, 2024 1 commit
- [Kernel] Add per-tensor and per-token AZP epilogues (#5941) · 8d59dbb0
  Luka Govedič authored Aug 06, 2024
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  8d59dbb0
05 Aug, 2024 1 commit
- [Kernel] Update CUTLASS to 3.5.1 (#7085) · 8571ac46
  Tyler Michael Smith authored Aug 05, 2024
  
  8571ac46
31 Jul, 2024 2 commits
- [Kernel] Tuned int8 Cutlass Kernels for SM75 (T4) (#6996) · 35e9c12b
  Varun Sundar Rabindranath authored Jul 31, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  35e9c12b
- [Kernel] Enable FP8 Cutlass for Ada Lovelace (#6950) · 93548eb3
  Varun Sundar Rabindranath authored Jul 31, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  93548eb3
30 Jul, 2024 1 commit
- [Kernel] Tuned int8 kernels for Ada Lovelace (#6848) · af647fb8
  Varun Sundar Rabindranath authored Jul 29, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  af647fb8
29 Jul, 2024 1 commit
- [Kernel] Tuned FP8 Kernels for Ada Lovelace (#6677) · 766435e6
  Varun Sundar Rabindranath authored Jul 29, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  766435e6
27 Jul, 2024 1 commit
- [Bug Fix] Illegal memory access, FP8 Llama 3.1 405b (#6852) · 55712941
  Lucas Wilkinson authored Jul 26, 2024
  
  55712941
14 Jul, 2024 1 commit
- [Kernel] Turn off CUTLASS scaled_mm for Ada Lovelace (#6384) · 9dad5cc8
  Tyler Michael Smith authored Jul 14, 2024
  
  9dad5cc8
28 Jun, 2024 1 commit
- [Bugfix] Fix compute datatype for cutlass 3.x epilogues (#5931) · 6a2d659d
  Tyler Michael Smith authored Jun 28, 2024
  
  6a2d659d
26 Jun, 2024 1 commit

[Kernel] Adding bias epilogue support for `cutlass_scaled_mm` (#5560) · 5bfd1bbc

Luka Govedič authored Jun 26, 2024


Co-authored-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

5bfd1bbc

23 Jun, 2024 1 commit
- [BugFix] [Kernel] Add Cutlass2x fallback kernels (#5744) · 6c916ac8
  Varun Sundar Rabindranath authored Jun 24, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  6c916ac8
20 Jun, 2024 3 commits
- [Bugfix] Fix the CUDA version check for FP8 support in the CUTLASS kernels (#5715) · 3f3b6b21
  Tyler Michael Smith authored Jun 20, 2024
  
  3f3b6b21
- [Kernel] Update Cutlass int8 kernel configs for SM80 (#5275) · a7dcc620
  Varun Sundar Rabindranath authored Jun 20, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  a7dcc620
- [Kernel] Update Cutlass int8 kernel configs for SM90 (#5514) · 111af1fa
  Varun Sundar Rabindranath authored Jun 20, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  111af1fa
14 Jun, 2024 1 commit
- [Kernel] Fix CUTLASS 3.x custom broadcast load epilogue (#5516) · 703475f6
  Tyler Michael Smith authored Jun 14, 2024
  
  703475f6
13 Jun, 2024 1 commit

[Kernel] Factor out epilogues from cutlass kernels (#5391) · 85657b56

Tyler Michael Smith authored Jun 13, 2024


Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: zifeitong <zifei.tong@parasail.io>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

85657b56

09 Jun, 2024 1 commit
- [Kernel][Misc] Use TORCH_LIBRARY instead of PYBIND11_MODULE for custom ops (#5047) · 5467ac31
  bnellnm authored Jun 09, 2024
  
  5467ac31
05 Jun, 2024 1 commit
- [Kernel] Add GPU architecture guards to the CUTLASS w8a8 kernels to reduce binary size (#5157) · ccd4f129
  Tyler Michael Smith authored Jun 05, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  ccd4f129
01 Jun, 2024 3 commits
- [Kernel] Update Cutlass fp8 configs (#5144) · f081c3ce
  Varun Sundar Rabindranath authored Jun 01, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  f081c3ce
- [Kernel] Refactor CUTLASS kernels to always take scales that reside on the GPU (#5137) · 260d119e
  Tyler Michael Smith authored Jun 01, 2024
  
  260d119e
- [Build] Guard against older CUDA versions when building CUTLASS 3.x kernels (#5168) · 1197e021
  Tyler Michael Smith authored May 31, 2024
  
  1197e021
22 May, 2024 2 commits
- [Kernel] Fixup for CUTLASS kernels in CUDA graphs (#4954) · 8674f988
  Tyler Michael Smith authored May 22, 2024
```
Pass the CUDA stream into the CUTLASS GEMMs, to avoid future issues with CUDA graphs
```
  8674f988
- [CI/Build] Enforce style for C++ and CUDA code with `clang-format` (#4722) · 5f6d10c1
  Michael Goin authored May 22, 2024
  
  5f6d10c1
16 May, 2024 1 commit
- [Kernel] Add w8a8 CUTLASS kernels (#4749) · 2060e936
  Tyler Michael Smith authored May 16, 2024
  
  2060e936