Commits · 8b8c209e352899c870fe348013a99a91262bf1e4 · OpenDAS / vllm_cscc

25 Jun, 2025 1 commit
- static_scaled_fp8_quant should not run when scale.numel is not 1 (#20076) · 8b8c209e
  Eldar Kurtić authored Jun 25, 2025
  
  8b8c209e
17 Jun, 2025 1 commit
- [Perf] Optimize `moe_align_block_size` CUDA kernel (#19572) · ffb2cd6b
  Wentao Ye authored Jun 17, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  ffb2cd6b
16 Jun, 2025 1 commit
- [Kernel] GGUF MMVQ kernel for multiple input vectors (#18754) · dec66d25
  Szymon Ożóg authored Jun 16, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
```
  dec66d25
12 Jun, 2025 4 commits
- [Bugfix] Enforce contiguous input for dynamic_per_token FP8/INT8 quant (#19452) · a3319f4f
  Michael Goin authored Jun 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a3319f4f
- [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) · f98548b9
  Luka Govedič authored Jun 12, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  f98548b9
- [AMD][Kernel][BugFix] fix test_rocm_compressed_tensors_w8a8 for rocm (#19509) · 2e090bd5
  rasmith authored Jun 12, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  2e090bd5
- [CI] change spell checker from codespell to typos (#18711) · 2f1c19b2
  Ning Xie authored Jun 12, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  2f1c19b2
07 Jun, 2025 1 commit

[Kernel] Integrate CUTLASS MoE kernel with PPLX (#18762) · 84166fee

ElizaWszola authored Jun 07, 2025


Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

84166fee

05 Jun, 2025 1 commit
- [Hardware][NVIDIA] FP4 MoE kernel optimization (#19110) · 61059bee
  Chiyue Wei authored Jun 05, 2025
```
Signed-off-by: Chiyue Wei <chiyuew@nvidia.com>
Co-authored-by: Chiyue Wei <chiyuew@nvidia.com>
```
  61059bee
04 Jun, 2025 1 commit
- [KERNEL] Sampler. CUDA kernel for applying repetition penalty (#18437) · 5d6d1adf
  Vadim Gimpelson authored Jun 04, 2025
  
  5d6d1adf
03 Jun, 2025 1 commit
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
23 May, 2025 1 commit

[ModelOpt] Introduce VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE env var to control... · f2036734

Pavani Majety authored May 23, 2025


[ModelOpt] Introduce VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE env var to control blockscale tensor allocation (#18160)
Signed-off-by: Pavani Majety <pmajety@nvidia.com>

f2036734

14 May, 2025 1 commit
- [Kernel] Have rotary embeddings support tensors (#18046) · d93c976a
  Lucas Wilkinson authored May 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d93c976a
13 May, 2025 1 commit
- Implements dual-chunk-flash-attn backend for dual chunk attention with sparse... · 60f76243
  Tao He authored May 13, 2025
```
Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844)
```
  60f76243
11 May, 2025 1 commit
- [Kernel] fp4 marlin kernel (#17687) · d74e5f37
  Jinzhen Lin authored May 11, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d74e5f37
09 May, 2025 1 commit
- [Hardware/NVIDIA/Kernel] Enable nvidia/DeepSeek-R1-FP4 Model (#16362) · 0c0fdae8
  Pavani Majety authored May 09, 2025
  
  0c0fdae8
07 May, 2025 4 commits
- [ROCm][FP8][Kernel] FP8 quantization fused into Custom Paged Attention (#17139) · 32aa74c0
  Gregory Shtrasberg authored May 07, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  32aa74c0
- Make key optional for rotary embedding (#17566) · 98c89e16
  Yong Hoon Shin authored May 07, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  98c89e16
- [Kernel] Use fused rmsnorm for some models like qwen3 series (#17735) · f80ae5bd
  Wanrui Dai authored May 07, 2025
```
Signed-off-by: evian <eviantai@u.nus.edu>
Co-authored-by: evian <eviantai@u.nus.edu>
```
  f80ae5bd
- [Kernel] GGUF MoeVec kernel (#16780) · 1a45a613
  Szymon Ożóg authored May 07, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  1a45a613
05 May, 2025 1 commit
- [Kernel] some optimizations for dense marlin and moe marlin (#16850) · 1d0c9d6b
  Jinzhen Lin authored May 06, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  1d0c9d6b
03 May, 2025 1 commit
- [Quantizaton] [AMD] Add support for running DeepSeek int8 w8a8 MoE on ROCm (#17558) · e3d0a1d1
  rasmith authored May 02, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  e3d0a1d1
29 Apr, 2025 1 commit
- [Bugfix] Add contiguous call inside rope kernel wrapper (#17091) · 17eb306f
  Zhengyuan Su (苏政渊) authored Apr 29, 2025
```
Signed-off-by: 苏政渊 <suzhengyuan@moonshot.cn>
Co-authored-by: 苏政渊 <suzhengyuan@moonshot.cn>
```
  17eb306f
27 Apr, 2025 1 commit
- [NVIDIA] Support Cutlass MLA for Blackwell GPUs (#16032) · ed7a29d9
  Kaixi Hou authored Apr 27, 2025
```
Signed-off-by: kaixih <kaixih@nvidia.com>
```
  ed7a29d9
22 Apr, 2025 1 commit
- [Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830) · 188b7f9b
  Charlie Fu authored Apr 21, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
```
  188b7f9b
15 Apr, 2025 1 commit

[Kernel] moe wna16 marlin kernel (#14447) · d06ba4ed

Jinzhen Lin authored Apr 15, 2025


Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

d06ba4ed

11 Apr, 2025 1 commit
- [Kernel] support merge_attn_states CUDA kernel, 3x speedup (#16173) · e9528f6d
  DefTruth authored Apr 11, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  e9528f6d
03 Apr, 2025 1 commit

[ROCM][KERNEL] Paged attention for V1 (#15720) · e73ff24e

Aleksandr Malyshev authored Apr 02, 2025


Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>

e73ff24e

02 Apr, 2025 1 commit
- [Kernel] Add more dtype support for GGUF dequantization (#15879) · 90969fb3
  LukasBluebaum authored Apr 02, 2025
```
Signed-off-by: lukas.bluebaum <lukas.bluebaum@aleph-alpha.com>
```
  90969fb3
01 Apr, 2025 2 commits
- Add option to use DeepGemm contiguous grouped gemm kernel for fused MoE operations. (#13932) · e59ca942
  bnellnm authored Apr 01, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  e59ca942
- [Distributed] Add custom allreduce support for ROCM (#14125) · b7b7676d
  Ilya Markov authored Apr 01, 2025
```
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
```
  b7b7676d
27 Mar, 2025 1 commit

[Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) · 9239bf71

ElizaWszola authored Mar 27, 2025


Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>

9239bf71

25 Mar, 2025 1 commit
- [Kernel][CPU] CPU MLA (#14744) · 4f044b1d
  Thien Tran authored Mar 25, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  4f044b1d
12 Mar, 2025 3 commits
- [ROCm][Bugfix] Ensure that the moe_wna16_gemm kernel is not built on ROCm platforms. (#14629) · 45f3f3f5
  Sage Moore authored Mar 12, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  45f3f3f5
- [Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) · debd6bbf
  Pavani Majety authored Mar 11, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  debd6bbf
- [Kernel] GGUF MoE kernel (#14613) · e22ee1e7
  Szymon Ożóg authored Mar 12, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
```
  e22ee1e7
11 Mar, 2025 2 commits
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
- [Kernel] moe wna16 cuda kernel (#13321) · 90e88ab7
  Jinzhen Lin authored Mar 11, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  90e88ab7
10 Mar, 2025 1 commit

[Kernel] Add more dtype support for GGUF kernels (#14043) · 89cdaa83

Szymon Ożóg authored Mar 10, 2025


Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>

89cdaa83

07 Mar, 2025 1 commit
- [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) · d0feea31
  Jinzhen Lin authored Mar 08, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d0feea31