Commits · 0e40ac9b7b5d953dfe38933bc7d2fb0a6c8da53c · OpenDAS / vllm_cscc

22 Sep, 2024 1 commit
- [ci][build] fix vllm-flash-attn (#8699) · 0e40ac9b
  youkaichao authored Sep 21, 2024
  
  0e40ac9b
21 Sep, 2024 1 commit
- [Kernel] Build flash-attn from source (#8245) · 71c60491
  Luka Govedič authored Sep 21, 2024
  
  71c60491
14 Sep, 2024 1 commit
- [Kernel][Hardware][Amd]Custom paged attention kernel for rocm (#8310) · 1ef0d2ef
  Charlie Fu authored Sep 13, 2024
  
  1ef0d2ef
10 Sep, 2024 1 commit
- [CI/Build][Kernel] Update CUTLASS to 3.5.1 tag (#8043) · 94144e72
  Tyler Michael Smith authored Sep 10, 2024
  
  94144e72
06 Sep, 2024 1 commit
- [Misc] Remove `SqueezeLLM` (#8220) · 23f32229
  Dipika Sikka authored Sep 06, 2024
  
  23f32229
28 Aug, 2024 1 commit
- [Kernel/Model] Migrate mamba_ssm and causal_conv1d kernels to vLLM (#7651) · fdd9daaf
  Mor Zusman authored Aug 29, 2024
  
  fdd9daaf
27 Aug, 2024 1 commit
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766) · fc911880
  Dipika Sikka authored Aug 27, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  fc911880
22 Aug, 2024 2 commits
- [Bugfix] Don't build machete on cuda <12.0 (#7757) · 55d63b12
  Lucas Wilkinson authored Aug 22, 2024
  
  55d63b12
- Revert "[Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527)" (#7764) · aae74ef9
  Michael Goin authored Aug 21, 2024
  
  aae74ef9
21 Aug, 2024 2 commits
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527) · 8678a69a
  Dipika Sikka authored Aug 21, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  8678a69a
- [Bugfix] Pass PYTHONPATH from setup.py to CMake (#7730) · 1b32e026
  sasha0552 authored Aug 21, 2024
  
  1b32e026
20 Aug, 2024 1 commit
- [Kernel] (1/N) Machete - Hopper Optimized Mixed Precision Linear Kernel (#7174) · 5288c06a
  Lucas Wilkinson authored Aug 20, 2024
  
  5288c06a
12 Aug, 2024 1 commit
- [CI/Build] bump minimum cmake version (#6999) · 774cd1d3
  Daniele authored Aug 13, 2024
  
  774cd1d3
05 Aug, 2024 3 commits
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
- [Bugfix][CI/Build] Fix CUTLASS FetchContent (#7171) · 4cf1dc39
  Tyler Michael Smith authored Aug 05, 2024
  
  4cf1dc39
- [Kernel] Update CUTLASS to 3.5.1 (#7085) · 8571ac46
  Tyler Michael Smith authored Aug 05, 2024
  
  8571ac46
02 Aug, 2024 2 commits
- [Misc] Disambiguate quantized types via a new ScalarType (#6396) · a8d604ca
  Lucas Wilkinson authored Aug 02, 2024
  
  a8d604ca
- [CI/Build] Add support for Python 3.12 (#7035) · b482b9a5
  Michael Goin authored Aug 02, 2024
  
  b482b9a5
01 Aug, 2024 3 commits
- [CI/Build][Bugfix] Fix CUTLASS header-only line (#7034) · 6a11fdfb
  Tyler Michael Smith authored Aug 01, 2024
  
  6a11fdfb
- [CI/Build] Update PyTorch to 2.4.0 (#6951) · 7e0861bd
  Sage Moore authored Aug 01, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7e0861bd
- [Kernel][RFC] Refactor the punica kernel based on Triton (#5036) · 7ecee343
  Jee Jee Li authored Aug 01, 2024
  
  7ecee343
31 Jul, 2024 1 commit
- Support W4A8 quantization for vllm (#5218) · 6512937d
  HandH1998 authored Jul 31, 2024
  
  6512937d
21 Jul, 2024 1 commit
- [Kernel][Core] Add AWQ support to the Marlin kernel (#6612) · 396d92d5
  Alexander Matveev authored Jul 21, 2024
  
  396d92d5
20 Jul, 2024 1 commit
- [Bugfix][CI/Build][Hardware][AMD] Fix AMD tests, add HF cache, update CK FA,... · 06d6c5fe
  Matt Wong authored Jul 20, 2024
```
[Bugfix][CI/Build][Hardware][AMD] Fix AMD tests, add HF cache, update CK FA, add partially supported model notes (#6543)
```
  06d6c5fe
17 Jul, 2024 1 commit
- [Core] draft_model_runner: Implement prepare_inputs on GPU for advance_step (#6338) · e76466dd
  Alexander Matveev authored Jul 17, 2024
  
  e76466dd
12 Jul, 2024 1 commit
- [MISC] Upgrade dependency to PyTorch 2.3.1 (#5327) · aa48e502
  Cody Yu authored Jul 12, 2024
  
  aa48e502
03 Jul, 2024 1 commit
- [Kernel] Expand FP8 support to Ampere GPUs using FP8 Marlin (#5975) · 47f0954a
  Michael Goin authored Jul 03, 2024
  
  47f0954a
26 Jun, 2024 1 commit

[Kernel] Adding bias epilogue support for `cutlass_scaled_mm` (#5560) · 5bfd1bbc

Luka Govedič authored Jun 26, 2024


Co-authored-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

5bfd1bbc

25 Jun, 2024 1 commit
- [Hardware][AMD][CI/Build][Doc] Upgrade to ROCm 6.1, Dockerfile improvements, test fixes (#5422) · dd793d1d
  Matt Wong authored Jun 25, 2024
  
  dd793d1d
13 Jun, 2024 1 commit

[Kernel] Factor out epilogues from cutlass kernels (#5391) · 85657b56

Tyler Michael Smith authored Jun 13, 2024


Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: zifeitong <zifei.tong@parasail.io>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

85657b56

09 Jun, 2024 1 commit
- [Kernel][Misc] Use TORCH_LIBRARY instead of PYBIND11_MODULE for custom ops (#5047) · 5467ac31
  bnellnm authored Jun 09, 2024
  
  5467ac31
02 Jun, 2024 1 commit
- [Kernel][ROCm][AMD] enable fused topk_softmax kernel for moe layer (#4927) · a66cf40b
  Divakar Verma authored Jun 02, 2024
```
This PR enables the fused topk_softmax kernel used in moe layer for HIP
```
  a66cf40b
31 May, 2024 1 commit
- Fix cutlass sm_90a vesrion in CMakeList · 533c2177
  simon-mo authored May 31, 2024
  
  533c2177
30 May, 2024 1 commit
- [Build] Disable sm_90a in cu11 (#5141) · 45a1a69b
  Simon Mo authored May 30, 2024
  
  45a1a69b
23 May, 2024 1 commit

[Kernel] Initial Activation Quantization Support (#4525) · a1242324

Dipika Sikka authored May 23, 2024


Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

a1242324

16 May, 2024 2 commits
- [Kernel] Add w8a8 CUTLASS kernels (#4749) · 2060e936
  Tyler Michael Smith authored May 16, 2024
  
  2060e936
- Add GPTQ Marlin 2:4 sparse structured support (#4790) · 6979ade3
  Alexander Matveev authored May 16, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  6979ade3
10 May, 2024 1 commit
- [Kernel] Refactor FP8 kv-cache with NVIDIA float8_e4m3 support (#4535) · c8331017
  Cody Yu authored May 09, 2024
  
  c8331017
09 May, 2024 1 commit
- [ROCm] Add support for Punica kernels on AMD GPUs (#3140) · ff5abcd7
  kliuae authored May 10, 2024
```
Co-authored-by: miloice <jeffaw99@hotmail.com>
```
  ff5abcd7
30 Apr, 2024 1 commit
- [Misc] Upgrade to `torch==2.3.0` (#4454) · d627a3d8
  Michael Goin authored Apr 29, 2024
  
  d627a3d8