Commits · 4fc5c23bb64719bda4f2b24e0b95637ccd530fab · OpenDAS / vllm_cscc

13 Feb, 2025 1 commit
- [NVIDIA] Support nvfp4 quantization (#12784) · 4fc5c23b
  Kaixi Hou authored Feb 12, 2025
  
  4fc5c23b
11 Feb, 2025 1 commit

[Build] Fix cuda link target of cumem_allocator in CPU env (#12863) · da317197

Yuhong Guo authored Feb 11, 2025


Signed-off-by: YuhongGuo <yuhong.gyh@antgroup.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

da317197

07 Feb, 2025 1 commit
- [Bugfix] FA2 illegal memory access (#12848) · ef533d25
  Lucas Wilkinson authored Feb 06, 2025
  
  ef533d25
31 Jan, 2025 1 commit
- [Kernel] Update `cutlass_scaled_mm` to support 2d group (blockwise) scaling (#11868) · 9798b2fb
  Lucas Wilkinson authored Jan 30, 2025
  
  9798b2fb
29 Jan, 2025 1 commit
- Revert "[Build/CI] Fix libcuda.so linkage" (#12552) · 73aa6cfd
  Tyler Michael Smith authored Jan 29, 2025
  
  73aa6cfd
27 Jan, 2025 1 commit
- [Build] Only build 9.0a for scaled_mm and sparse kernels (#12339) · 103bd17a
  Lucas Wilkinson authored Jan 27, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  103bd17a
26 Jan, 2025 2 commits
- [Build/CI] Fix libcuda.so linkage (#12424) · 72bac730
  Tyler Michael Smith authored Jan 26, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  72bac730
- [Bugfix][Kernel] Fix perf regression caused by PR #12405 (#12434) · 68f11149
  Lucas Wilkinson authored Jan 26, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  68f11149
24 Jan, 2025 2 commits

[Bugfix][Kernel] FA3 Fix - RuntimeError: This flash attention build only... · 3132a933

Lucas Wilkinson authored Jan 24, 2025


[Bugfix][Kernel] FA3 Fix - RuntimeError: This flash attention build only supports pack_gqa (for build size reasons). (#12405)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

3132a933

[Bugfix][Kernel] Fix CUDA 11.8 being broken by FA3 build (#12375) · ab5bbf5a
Lucas Wilkinson authored Jan 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
ab5bbf5a

23 Jan, 2025 1 commit
- [Kernel] Flash Attention 3 Support (#12093) · 978b45f3
  Lucas Wilkinson authored Jan 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  978b45f3
22 Jan, 2025 1 commit
- [Core] Support fully transparent sleep mode (#11743) · 68ad4e3a
  youkaichao authored Jan 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  68ad4e3a
01 Jan, 2025 1 commit
- [V1] Implement Cascade Attention (#11635) · 73001445
  Woosuk Kwon authored Jan 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  73001445
30 Dec, 2024 1 commit
- [Build][Kernel] Update CUTLASS to v3.6.0 (#11607) · 970d6d07
  Tyler Michael Smith authored Dec 30, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  970d6d07
19 Dec, 2024 1 commit
- [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
  Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  5a9da2e6
18 Dec, 2024 1 commit

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd

13 Dec, 2024 1 commit

[torch.compile] Dynamic fp8 + rms_norm fusion (#10906) · 30870b4f

Luka Govedič authored Dec 12, 2024


Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

30870b4f

02 Dec, 2024 1 commit
- [Kernel] Use `out` arg in flash_attn_varlen_func (#10811) · 073a4bd1
  Woosuk Kwon authored Dec 01, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  073a4bd1
28 Nov, 2024 2 commits
- [Kernel] Update vllm-flash-attn version to reduce CPU overheads (#10742) · 8c1e77fb
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8c1e77fb
- [Kernel] Update vllm-flash-attn version (#10736) · 9a8bff02
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  9a8bff02
26 Nov, 2024 1 commit
- [Hardware][NVIDIA] Add non-NVML CUDA mode for Jetson (#9735) · f5792c7c
  Conroy Cheers authored Nov 27, 2024
```
Signed-off-by: Conroy Cheers <conroy@corncheese.org>
```
  f5792c7c
23 Nov, 2024 1 commit
- [AMD] Add support for GGUF quantization on ROCm (#10254) · 7c25fe45
  kliuae authored Nov 23, 2024
  
  7c25fe45
20 Nov, 2024 1 commit
- [CI/Build] Support compilation with local cutlass path (#10423) (#10424) · 7629a9c6
  wchen61 authored Nov 20, 2024
  
  7629a9c6
12 Nov, 2024 1 commit

Splitting attention kernel file (#10091) · 812c981f

Aleksandr Malyshev authored Nov 11, 2024


Signed-off-by: maleksan85 <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>

812c981f

08 Nov, 2024 1 commit
- [torch.compile] Fuse RMSNorm with quant (#9138) · 4f93dfe9
  Luka Govedič authored Nov 08, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  4f93dfe9
06 Nov, 2024 2 commits

[CI/Build] Drop Python 3.8 support (#10038) · 098f94de

Russell Bryant authored Nov 06, 2024


Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

098f94de

[CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
21063c11

04 Nov, 2024 1 commit
- [Bugfix] Upgrade to pytorch 2.5.1 (#10001) · d93478b3
  bnellnm authored Nov 04, 2024
```
Signed-off-by: Bill Nell <bill@neuralmagic.com>
```
  d93478b3
27 Oct, 2024 1 commit

[Misc] Upgrade to pytorch 2.5 (#9588) · 3cb07a36

bnellnm authored Oct 27, 2024


Signed-off-by: Bill Nell <bill@neuralmagic.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

3cb07a36

24 Oct, 2024 1 commit
- [Performance][Kernel] Fused_moe Performance Improvement (#9384) · 59449095
  Charlie Fu authored Oct 24, 2024
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  59449095
23 Oct, 2024 1 commit
- [Build] Fix `FetchContent` multiple build issue (#9596) · 51c24c97
  Luka Govedič authored Oct 23, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  51c24c97
22 Oct, 2024 1 commit
- [Bugfix] Fix spurious "No compiled cutlass_scaled_mm ..." for W8A8 on Turing (#9487) · d1e82408
  Lucas Wilkinson authored Oct 22, 2024
  
  d1e82408
17 Oct, 2024 1 commit
- [Bugfix] Fix support for dimension like integers and ScalarType (#9299) · eca2c5f7
  bnellnm authored Oct 17, 2024
  
  eca2c5f7
16 Oct, 2024 1 commit
- [Bugfix][CI/Build] Fix CUDA 11.8 Build (#9386) · 717a5f82
  Lucas Wilkinson authored Oct 15, 2024
  
  717a5f82
11 Oct, 2024 1 commit
- [Bugfix][CI/Build] Fix docker build where CUDA archs < 7.0 are being detected (#9254) · de9fb4be
  Lucas Wilkinson authored Oct 11, 2024
  
  de9fb4be
04 Oct, 2024 3 commits
- [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) · 05d68643
  ElizaWszola authored Oct 04, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika Sikka <ds3822@columbia.edu>
```
  05d68643
- [Bugfix] Flash attention arches not getting set properly (#9062) · 22482e49
  Lucas Wilkinson authored Oct 04, 2024
  
  22482e49
- [CI/Build] Per file CUDA Archs (improve wheel size and dev build times) (#8845) · aeb37c2a
  Lucas Wilkinson authored Oct 03, 2024
  
  aeb37c2a
29 Sep, 2024 1 commit
- [Build/CI] Set FETCHCONTENT_BASE_DIR to one location for better caching (#8930) · 2e7fe7e7
  Tyler Michael Smith authored Sep 28, 2024
  
  2e7fe7e7
24 Sep, 2024 1 commit
- [Kernel] Split Marlin MoE kernels into multiple files (#8661) · a928ded9
  ElizaWszola authored Sep 24, 2024
```
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  a928ded9