Commits · c2170a5b395acb9f5f4ce8425c3be18aacb67513 · OpenDAS / vllm_cscc · GitLab

08 Nov, 2024 1 commit
- [torch.compile] Fuse RMSNorm with quant (#9138) · 4f93dfe9
  Luka Govedič authored Nov 08, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  4f93dfe9
06 Nov, 2024 1 commit
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
22 Oct, 2024 1 commit
- [Bugfix] Fix spurious "No compiled cutlass_scaled_mm ..." for W8A8 on Turing (#9487) · d1e82408
  Lucas Wilkinson authored Oct 22, 2024
  
  d1e82408
17 Oct, 2024 2 commits
- [Bugfix] Fix support for dimension like integers and ScalarType (#9299) · eca2c5f7
  bnellnm authored Oct 17, 2024
  
  eca2c5f7
- [BugFix] [Kernel] Fix GPU SEGV occurring in int8 kernels (#9391) · 92d86da2
  rasmith authored Oct 16, 2024
  
  92d86da2
16 Oct, 2024 1 commit
- [Bugfix][Kernel] Prevent integer overflow in fp8 dynamic per-token quantize kernel (#9425) · c3fab5f7
  Tyler Michael Smith authored Oct 16, 2024
  
  c3fab5f7
10 Oct, 2024 2 commits
- [Bugfix] Fix Machete unittests failing with `NotImplementedError` (#9218) · 18511aed
  Lucas Wilkinson authored Oct 10, 2024
  
  18511aed
- [Bugfix] Machete garbage results for some models (large K dim) (#9212) · a64e7b94
  Lucas Wilkinson authored Oct 10, 2024
  
  a64e7b94
04 Oct, 2024 2 commits
- [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) · 05d68643
  ElizaWszola authored Oct 04, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika Sikka <ds3822@columbia.edu>
```
  05d68643
- [CI/Build] Per file CUDA Archs (improve wheel size and dev build times) (#8845) · aeb37c2a
  Lucas Wilkinson authored Oct 03, 2024
  
  aeb37c2a
01 Oct, 2024 1 commit
- [CI/Build] Fix machete generated kernel files ordering (#8976) · aaccca2b
  Kevin H. Luu authored Sep 30, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  aaccca2b
23 Sep, 2024 1 commit

[Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701) · 86e9c8df

Lucas Wilkinson authored Sep 23, 2024

Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

86e9c8df

16 Sep, 2024 2 commits
- [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) · 5d73ae49
  Luka Govedič authored Sep 16, 2024
  
  5d73ae49
- [Bugfix][Kernel] Fix build for sm_60 in GGUF kernel (#8506) · 781e3b9a
  sasha0552 authored Sep 16, 2024
  
  781e3b9a
15 Sep, 2024 1 commit
- [Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357) · fc990f97
  Isotr0py authored Sep 16, 2024
  
  fc990f97
11 Sep, 2024 1 commit
- [Kernel][Misc] register ops to prevent graph breaks (#6917) · 73202dbe
  bnellnm authored Sep 11, 2024
```
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  73202dbe
06 Sep, 2024 1 commit
- [Misc] Remove `SqueezeLLM` (#8220) · 23f32229
  Dipika Sikka authored Sep 06, 2024
  
  23f32229
22 Aug, 2024 2 commits
- [Bugfix] Don't build machete on cuda <12.0 (#7757) · 55d63b12
  Lucas Wilkinson authored Aug 22, 2024
  
  55d63b12
- [Kernel] Replaced `blockReduce[...]` functions with `cub::BlockReduce` (#7233) · 7937009a
  Luka Govedič authored Aug 21, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7937009a
20 Aug, 2024 1 commit
- [Kernel] (1/N) Machete - Hopper Optimized Mixed Precision Linear Kernel (#7174) · 5288c06a
  Lucas Wilkinson authored Aug 20, 2024
  
  5288c06a
16 Aug, 2024 2 commits
- [Kernel] fix types used in aqlm and ggml kernels to support dynamo (#7596) · 37fd47e7
  bnellnm authored Aug 16, 2024
  
  37fd47e7
- [Feature][Hardware][Amd] Add fp8 Linear Layer for Rocm (#7210) · e837b624
  Charlie Fu authored Aug 16, 2024
  
  e837b624
12 Aug, 2024 1 commit
- [Misc] Use scalar type to dispatch to different `gptq_marlin` kernels (#7323) · 6aa33cb2
  Lucas Wilkinson authored Aug 12, 2024
  
  6aa33cb2
06 Aug, 2024 1 commit
- [Kernel] Add per-tensor and per-token AZP epilogues (#5941) · 8d59dbb0
  Luka Govedič authored Aug 06, 2024
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  8d59dbb0
05 Aug, 2024 3 commits
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
- [CI/Build] Suppress divide-by-zero and missing return statement warnings (#7001) · 6e4852ce
  Tyler Michael Smith authored Aug 05, 2024
  
  6e4852ce
- [Kernel] Update CUTLASS to 3.5.1 (#7085) · 8571ac46
  Tyler Michael Smith authored Aug 05, 2024
  
  8571ac46
02 Aug, 2024 1 commit
- [Misc] Disambiguate quantized types via a new ScalarType (#6396) · a8d604ca
  Lucas Wilkinson authored Aug 02, 2024
  
  a8d604ca
31 Jul, 2024 3 commits
- [Kernel] Tuned int8 Cutlass Kernels for SM75 (T4) (#6996) · 35e9c12b
  Varun Sundar Rabindranath authored Jul 31, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  35e9c12b
- [Kernel] Enable FP8 Cutlass for Ada Lovelace (#6950) · 93548eb3
  Varun Sundar Rabindranath authored Jul 31, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  93548eb3
- Support W4A8 quantization for vllm (#5218) · 6512937d
  HandH1998 authored Jul 31, 2024
  
  6512937d
30 Jul, 2024 4 commits
- [Kernel] Squash a few more warnings (#6914) · cbbc9044
  Tyler Michael Smith authored Jul 30, 2024
  
  cbbc9044
- [Kernel] Tuned int8 kernels for Ada Lovelace (#6848) · af647fb8
  Varun Sundar Rabindranath authored Jul 29, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  af647fb8
- [Kernel] Fix marlin divide-by-zero warnings (#6904) · 61a97c32
  Tyler Michael Smith authored Jul 29, 2024
  
  61a97c32
- [Kernel] Remove unused variables in awq/gemm_kernels.cu (#6908) · aae6d36f
  Tyler Michael Smith authored Jul 29, 2024
  
  aae6d36f
29 Jul, 2024 2 commits
- [Kernel] Fix deprecation function warnings squeezellm quant_cuda_kernel (#6901) · 60d1c6e5
  Tyler Michael Smith authored Jul 29, 2024
  
  60d1c6e5
- [Kernel] Tuned FP8 Kernels for Ada Lovelace (#6677) · 766435e6
  Varun Sundar Rabindranath authored Jul 29, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  766435e6
27 Jul, 2024 2 commits
- [Kernel] Increase precision of GPTQ/AWQ Marlin kernel (#6795) · 75acdaa4
  Alexander Matveev authored Jul 27, 2024
  
  75acdaa4
- [Bug Fix] Illegal memory access, FP8 Llama 3.1 405b (#6852) · 55712941
  Lucas Wilkinson authored Jul 26, 2024
  
  55712941
26 Jul, 2024 1 commit
- [Bugfix][Kernel] Promote another index to int64_t (#6838) · 50704f52
  Tyler Michael Smith authored Jul 26, 2024
  
  50704f52