Commits · 9239bf718e5ebb5ab871ac8ed09fb80ed02fa82b · OpenDAS / vllm_cscc

27 Mar, 2025 1 commit

[Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) · 9239bf71

ElizaWszola authored Mar 27, 2025


Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>

9239bf71

25 Mar, 2025 3 commits

[Kernel] Fix conflicting macro names for gguf kernels (#15456) · a6081600
Szymon Ożóg authored Mar 25, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
```
a6081600
[Kernel][CPU] CPU MLA (#14744) · 4f044b1d
Thien Tran authored Mar 25, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
4f044b1d

Fix CUDA kernel index data type in... · 051da7ef

Lu Fang authored Mar 25, 2025

Fix CUDA kernel index data type in vllm/csrc/quantization/gptq_marlin/awq_marlin_repack.cu +10 (#15160)
Signed-off-by: Lu Fang <lufang@fb.com>
Co-authored-by: Richard Barnes <rbarnes@meta.com>

051da7ef

24 Mar, 2025 1 commit
- [Kernel] allow non-contiguous input for marlin kernel (#14658) · 6b3cc75b
  Jinzhen Lin authored Mar 24, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  6b3cc75b
21 Mar, 2025 1 commit

Fix CUDA kernel index data type in... · d3ccbd63

Lu Fang authored Mar 20, 2025

Fix CUDA kernel index data type in vllm/csrc/quantization/fused_kernels/layernorm_utils.cuh +10 (#15159)
Signed-off-by: Lu Fang <lufang@fb.com>
Co-authored-by: Richard Barnes <rbarnes@meta.com>

d3ccbd63

18 Mar, 2025 1 commit
- [Misc][Docs] fix the comments of KV_T and CACHE_T in CALL_RESHAPE_AND_CACHE_XX macros (#14347) · 64fc2193
  Serena authored Mar 18, 2025
  
  64fc2193
17 Mar, 2025 1 commit
- [MISC] More AMD unused var clean up (#14926) · cd0cd851
  Lu Fang authored Mar 17, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  cd0cd851
15 Mar, 2025 2 commits
- [CPU] Support FP8 KV cache (#14741) · a2ae4965
  Li, Jiang authored Mar 15, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a2ae4965
- [Misc][Easy] Annotate unused vars in the csrc files (#14798) · 8c0d15d5
  Lu Fang authored Mar 14, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  8c0d15d5
14 Mar, 2025 4 commits
- [Bugfix][Kernel]: Fix AllSpark kernel compilation errors and enable for CUDA < 12.0 (#14430) · 977a1677
  Yajie Wang authored Mar 15, 2025
```
Signed-off-by: wyj371990 <wyj371990@alibaba-inc.com>
```
  977a1677
- [Bugfix][W8A8] fixed cutlass block fp8 binding (#14796) · 40253bab
  DefTruth authored Mar 14, 2025
  
  40253bab
- [Bugfix][Kernel][CPU] Fix num_tokens in CPU rotary embedding kernel (#14667) · 27b50f1f
  Thien Tran authored Mar 14, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  27b50f1f
- forward fix PR 14245, restore build on ROCm 6.2 (#14709) · 2a602b05
  Jeff Daily authored Mar 13, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  2a602b05
12 Mar, 2025 4 commits
- [FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash... · 916836bb
  TJian authored Mar 13, 2025
```
[FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash Attention to enable embedding models. (#14664)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  916836bb
- [ROCm][Bugfix] Ensure that the moe_wna16_gemm kernel is not built on ROCm platforms. (#14629) · 45f3f3f5
  Sage Moore authored Mar 12, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  45f3f3f5
- [Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) · debd6bbf
  Pavani Majety authored Mar 11, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  debd6bbf
- [Kernel] GGUF MoE kernel (#14613) · e22ee1e7
  Szymon Ożóg authored Mar 12, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
```
  e22ee1e7
11 Mar, 2025 3 commits
- [BugFix/Build] Fix sparse kernels not getting built on hopper (#14572) · 07b4b7a3
  Lucas Wilkinson authored Mar 11, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  07b4b7a3
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
- [Kernel] moe wna16 cuda kernel (#13321) · 90e88ab7
  Jinzhen Lin authored Mar 11, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  90e88ab7
10 Mar, 2025 1 commit

[Kernel] Add more dtype support for GGUF kernels (#14043) · 89cdaa83

Szymon Ożóg authored Mar 10, 2025


Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>

89cdaa83

08 Mar, 2025 1 commit

[Build/BugFix] Fix hopper 12.8 build (#14354) · 7caff01a

Lucas Wilkinson authored Mar 08, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

7caff01a

07 Mar, 2025 2 commits
- [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) · d0feea31
  Jinzhen Lin authored Mar 08, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d0feea31
- [BugFix] Illegal Memory Access in the blockwise cutlass fp8 GEMMs (#14396) · e5e03c2c
  Lucas Wilkinson authored Mar 07, 2025
  
  e5e03c2c
06 Mar, 2025 2 commits
- [Kernel] Add needs_fixed_stride_order tag to most GEMMs (#14306) · 99b0915d
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  99b0915d
- Adding cpu inference with VXE ISA for s390x architecture (#12613) · ada19210
  Dilip Gowda Bhagavan authored Mar 06, 2025
```
Signed-off-by: Dilip Gowda Bhagavan <dilip.bhagavan@ibm.com>
Signed-off-by: Rishika Kedia <rishika.kedia@in.ibm.com>
Co-authored-by: Rishika Kedia <rishika.kedia@in.ibm.com>
```
  ada19210
04 Mar, 2025 1 commit
- add cutlass support for blackwell fp8 gemm (#13798) · f89978ad
  kushanam authored Mar 04, 2025
  
  f89978ad
03 Mar, 2025 3 commits
- [ROCm] Faster Custom Paged Attention kernels (#12348) · 848a6438
  TJian authored Mar 04, 2025
  
  848a6438
- [Bugfix] Explicitly include "omp.h" for MacOS to avoid installation failure (#14051) · 09e56f92
  Sheng Yao authored Mar 03, 2025
  
  09e56f92
- Update deprecated Python 3.8 typing (#13971) · cf069aa8
  Harry Mellor authored Mar 03, 2025
  
  cf069aa8
01 Mar, 2025 1 commit
- [Misc][Kernel]: Add GPTQAllSpark Quantization (#12931) · 6a92ff93
  YajieWang authored Mar 01, 2025
  
  6a92ff93
27 Feb, 2025 2 commits
- [ROCm][V1] Update reshape_and_cache to properly work with CUDA graph padding (#13922) · 378b3ef6
  Sage Moore authored Feb 26, 2025
  
  378b3ef6
- [ROCm][Quantization][Kernel] Use FP8 FNUZ when OCP flag is 0 or undefined (#13851) · a31614e3
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Feb 27, 2025
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  a31614e3
26 Feb, 2025 1 commit
- [Kernel][Build/CI] Bump CUTLASS to 3.8 and add initializers for cutlass epilogues (#13797) · 094b7d94
  Henry Tsang authored Feb 25, 2025
  
  094b7d94
25 Feb, 2025 1 commit
- [ROCm][Quantization][Kernel] Using HIP FP8 header (#12593) · aabeb268
  Gregory Shtrasberg authored Feb 25, 2025
  
  aabeb268
23 Feb, 2025 1 commit
- [CI/Build] Fix pre-commit errors from #13571 (#13709) · 82e0d601
  Roger Wang authored Feb 22, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  82e0d601
22 Feb, 2025 1 commit
- [NVIDIA] Support nvfp4 cutlass gemm (#13571) · e109e598
  Kaixi Hou authored Feb 22, 2025
  
  e109e598
21 Feb, 2025 2 commits

[Attention] MLA with chunked prefill (#12639) · 288cc6c2

Lucas Wilkinson authored Feb 21, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

288cc6c2

[Kernel]Add streamK for block-quantized CUTLASS kernels (#12978) · 839b27c6
leoneo authored Feb 21, 2025

839b27c6