Commits · debd6bbf0951222c9ad7a1a91b958027f2ed0782 · OpenDAS / vllm_cscc

12 Mar, 2025 2 commits
- [Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) · debd6bbf
  Pavani Majety authored Mar 11, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  debd6bbf
- [Kernel] GGUF MoE kernel (#14613) · e22ee1e7
  Szymon Ożóg authored Mar 12, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
```
  e22ee1e7
11 Mar, 2025 3 commits
- [BugFix/Build] Fix sparse kernels not getting built on hopper (#14572) · 07b4b7a3
  Lucas Wilkinson authored Mar 11, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  07b4b7a3
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
- [Kernel] moe wna16 cuda kernel (#13321) · 90e88ab7
  Jinzhen Lin authored Mar 11, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  90e88ab7
10 Mar, 2025 1 commit

[Kernel] Add more dtype support for GGUF kernels (#14043) · 89cdaa83

Szymon Ożóg authored Mar 10, 2025


Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>

89cdaa83

08 Mar, 2025 1 commit

[Build/BugFix] Fix hopper 12.8 build (#14354) · 7caff01a

Lucas Wilkinson authored Mar 08, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

7caff01a

07 Mar, 2025 2 commits
- [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) · d0feea31
  Jinzhen Lin authored Mar 08, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d0feea31
- [BugFix] Illegal Memory Access in the blockwise cutlass fp8 GEMMs (#14396) · e5e03c2c
  Lucas Wilkinson authored Mar 07, 2025
  
  e5e03c2c
06 Mar, 2025 2 commits
- [Kernel] Add needs_fixed_stride_order tag to most GEMMs (#14306) · 99b0915d
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  99b0915d
- Adding cpu inference with VXE ISA for s390x architecture (#12613) · ada19210
  Dilip Gowda Bhagavan authored Mar 06, 2025
```
Signed-off-by: Dilip Gowda Bhagavan <dilip.bhagavan@ibm.com>
Signed-off-by: Rishika Kedia <rishika.kedia@in.ibm.com>
Co-authored-by: Rishika Kedia <rishika.kedia@in.ibm.com>
```
  ada19210
04 Mar, 2025 1 commit
- add cutlass support for blackwell fp8 gemm (#13798) · f89978ad
  kushanam authored Mar 04, 2025
  
  f89978ad
03 Mar, 2025 3 commits
- [ROCm] Faster Custom Paged Attention kernels (#12348) · 848a6438
  TJian authored Mar 04, 2025
  
  848a6438
- [Bugfix] Explicitly include "omp.h" for MacOS to avoid installation failure (#14051) · 09e56f92
  Sheng Yao authored Mar 03, 2025
  
  09e56f92
- Update deprecated Python 3.8 typing (#13971) · cf069aa8
  Harry Mellor authored Mar 03, 2025
  
  cf069aa8
01 Mar, 2025 1 commit
- [Misc][Kernel]: Add GPTQAllSpark Quantization (#12931) · 6a92ff93
  YajieWang authored Mar 01, 2025
  
  6a92ff93
27 Feb, 2025 2 commits
- [ROCm][V1] Update reshape_and_cache to properly work with CUDA graph padding (#13922) · 378b3ef6
  Sage Moore authored Feb 26, 2025
  
  378b3ef6
- [ROCm][Quantization][Kernel] Use FP8 FNUZ when OCP flag is 0 or undefined (#13851) · a31614e3
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Feb 27, 2025
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  a31614e3
26 Feb, 2025 1 commit
- [Kernel][Build/CI] Bump CUTLASS to 3.8 and add initializers for cutlass epilogues (#13797) · 094b7d94
  Henry Tsang authored Feb 25, 2025
  
  094b7d94
25 Feb, 2025 1 commit
- [ROCm][Quantization][Kernel] Using HIP FP8 header (#12593) · aabeb268
  Gregory Shtrasberg authored Feb 25, 2025
  
  aabeb268
23 Feb, 2025 1 commit
- [CI/Build] Fix pre-commit errors from #13571 (#13709) · 82e0d601
  Roger Wang authored Feb 22, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  82e0d601
22 Feb, 2025 1 commit
- [NVIDIA] Support nvfp4 cutlass gemm (#13571) · e109e598
  Kaixi Hou authored Feb 22, 2025
  
  e109e598
21 Feb, 2025 4 commits
- [Attention] MLA with chunked prefill (#12639) · 288cc6c2
  Lucas Wilkinson authored Feb 21, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  288cc6c2
- [Kernel]Add streamK for block-quantized CUTLASS kernels (#12978) · 839b27c6
  leoneo authored Feb 21, 2025
  
  839b27c6
- Missing comment explaining VDR variable in GGUF kernels (#13290) · 1cdc8861
  Szymon Ożóg authored Feb 21, 2025
  
  1cdc8861
- [NVIDIA] Fix an issue to use current stream for the nvfp4 quant (#13632) · 27a09dc5
  Kaixi Hou authored Feb 20, 2025
  
  27a09dc5
20 Feb, 2025 1 commit
- [ROCm] MI300A compile targets deprecation (#13560) · 0023cd2b
  Gregory Shtrasberg authored Feb 20, 2025
  
  0023cd2b
15 Feb, 2025 1 commit
- [Bugfix][AMD] Update torch_bindings so that scaled_fp4_quant isn't build on ROCm (#13235) · c9f9d5b3
  Sage Moore authored Feb 14, 2025
  
  c9f9d5b3
14 Feb, 2025 2 commits
- [Kernel] Fix awq error when n is not divisable by 128 (#13227) · 8c32b08a
  Jinzhen Lin authored Feb 14, 2025
  
  8c32b08a
- [Kernel][Bugfix] Refactor and Fix CUTLASS 2:4 Sparse Kernels (#13198) · c1e37bf7
  Tyler Michael Smith authored Feb 13, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  c1e37bf7
13 Feb, 2025 2 commits
- Optimize moe_align_block_size for deepseek_v3 (#12850) · 2344192a
  Michael Goin authored Feb 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  2344192a
- [NVIDIA] Support nvfp4 quantization (#12784) · 4fc5c23b
  Kaixi Hou authored Feb 12, 2025
  
  4fc5c23b
12 Feb, 2025 1 commit
- [Misc] AMD Build Improvements (#12923) · f1042e86
  Shiyan Deng authored Feb 12, 2025
  
  f1042e86
10 Feb, 2025 1 commit
- [core] improve error handling when wake up from sleep mode (#12981) · 59fff4a0
  youkaichao authored Feb 10, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  59fff4a0
08 Feb, 2025 1 commit
- [CI/Build] Auto-fix Markdown files (#12941) · 8a69e0e2
  Cyrus Leung authored Feb 08, 2025
  
  8a69e0e2
06 Feb, 2025 2 commits
- [Kernel] Make rotary_embedding ops more flexible with input shape (#12777) · 85ac82d2
  Isotr0py authored Feb 07, 2025
  
  85ac82d2
- [ROCm][Kernel] Using the correct warp_size value · 5b19b930
  Gregory Shtrasberg authored Feb 05, 2025
  
  5b19b930
05 Feb, 2025 1 commit

[Perf] Mem align KV caches for CUDA devices (MLA perf improvement) (#12676) · 75e94309

Lucas Wilkinson authored Feb 04, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Lucas Wilkinson <lcwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

75e94309

03 Feb, 2025 2 commits

[Bugfix][Kernel] Fix per-token/per-channel quantization for Hopper scaled mm (#12696) · c11de33d
Tyler Michael Smith authored Feb 03, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
c11de33d

[Kernel] port sgl moe_align_block_size kernels (#12574) · 95460fc5

Yang Chen authored Feb 02, 2025

sgl_moe_align_block_size is based on:


https://github.com/sgl-project/sglang/commit/ded9fcd09a43d5e7d5bb31a2bc3e9fc21bf65d2a

moe_align_block_size is based on:


https://github.com/sgl-project/sglang/commit/ba5112ff691d791a9e38c6c71f59324a5fcb49d0

Signed-off-by: Yang Chen <yangche@fb.com>

95460fc5