Commits · 7961486a9b749b1b60d8b6fd5fb7d61596a9b041 · OpenDAS / vllm_cscc

09 Mar, 2026 1 commit

[Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA... · 580864d8

Roberto L. Castro authored Mar 09, 2026


[Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2 (#34917)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>

580864d8

02 Mar, 2026 1 commit

[Kernel] Integrate SM100 MXFP8 blockscaled grouped MM and quant kernels (#34448) · cb21972a

EdalatiAli authored Mar 02, 2026


Signed-off-by: EdalatiAli <aliedalati@cohere.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

cb21972a

01 Mar, 2026 1 commit
- [Mamba1] - Kernel Level Chunk Alignment for Prefix Caching (#34798) · bbf81f9a
  Asaf Gardin authored Mar 01, 2026
```
Signed-off-by: Josephasafg <ajgard7@gmail.com>
```
  bbf81f9a
26 Feb, 2026 1 commit

[WideEP] Remove pplx all2all backend (#33724) · eb19955c

Tyler Michael Smith authored Feb 26, 2026


Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

eb19955c

24 Feb, 2026 1 commit
- [Bugfix] Fix DSV3 kernels breaking _C and _moe_C on unsupported arches (#35123) · 3ef9fd0f
  Michael Goin authored Feb 23, 2026
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  3ef9fd0f
18 Feb, 2026 2 commits
- [Model Bash] DeepSeek R1 BF16 Min Latency QKV A GEMM (0.5% E2E Speedup) (#34758) · 6874638b
  Robert Shaw authored Feb 18, 2026
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
```
  6874638b
- [Bugfix] Fix quant RMS norm fusion for quantization with TMA-aligned scales (#33255) · a88b3be7
  ElizaWszola authored Feb 18, 2026
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  a88b3be7
10 Feb, 2026 1 commit

[Perf][Kernel] Add faster topKperRow decode kernel for DeepSeek-V3.2 sparse attention (#33680) · afdce12c

Roberto L. Castro authored Feb 10, 2026

Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

afdce12c

28 Jan, 2026 1 commit

[Quantization][Deprecation] Remove Marlin 24 (#32688) · af9b69f9

Robert Shaw authored Jan 28, 2026


Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

af9b69f9

25 Jan, 2026 1 commit
- [Perf][Kernel] Optimize FP4 quantization kernels (SM100F) (#32520) · fcb9df99
  Roberto L. Castro authored Jan 25, 2026
```
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
```
  fcb9df99
23 Jan, 2026 1 commit
- [Refactor] Rename `gptq_marlin` to `marlin` to match MoE (#32952) · 4561f139
  Michael Goin authored Jan 23, 2026
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  4561f139
22 Jan, 2026 1 commit
- OffloadingConnector: Support kernel_block_size != block_size (#30692) · 421012b6
  Or Ozeri authored Jan 22, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  421012b6
18 Jan, 2026 1 commit
- [Refactor] Remove unused cutlass moe problem size function (#32047) · eebc58df
  Wentao Ye authored Jan 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  eebc58df
10 Jan, 2026 1 commit

Fuse RoPE and MLA KV-cache write (#25774) · 80fead8b

PatrykSaffer authored Jan 10, 2026


Signed-off-by: Patryk Saffer <patryk.saffer99@gmail.com>
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai>
Co-authored-by: Patryk Saffer <patryk.saffer99@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

80fead8b

09 Jan, 2026 3 commits

[Quant] Make static quant support all group shapes (#30833) · 0a0aa077
Lucas Wilkinson authored Jan 09, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
0a0aa077

[Perf] Optimize cutlass moe problem size calculation, 5.3% E2E Throughput... · 308feab3

Wentao Ye authored Jan 09, 2026


[Perf] Optimize cutlass moe problem size calculation, 5.3% E2E Throughput improvement, 2.2% TTFT improvement (#31830)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

308feab3

[Perf][Kernel] Fused SiLU+Mul+Quant kernel for NVFP4 cutlass_moe (#31832) · 34cd32fe
Michael Goin authored Jan 09, 2026
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
34cd32fe

24 Dec, 2025 1 commit
- [Misc] Remove unused custom ops `copy_blocks` and `copy_blocks_mla` (#30967) · 4ed11105
  rongfu.leng authored Dec 24, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  4ed11105
19 Dec, 2025 1 commit
- [MoE Refactor][3/N] Deprecate cutlass block quant fp8 (b200) (#30990) · 83a317f6
  Robert Shaw authored Dec 19, 2025
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
```
  83a317f6
12 Dec, 2025 1 commit
- [Attention] Use sparse prefill kernel for fp8 kv-cache in DeepSeek-v3.2 (#27532) · 3e41992f
  Lucas Wilkinson authored Dec 12, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  3e41992f
09 Dec, 2025 1 commit
- [Kernel]Support W4A8 Grouped GEMM on Hopper (#29691) · f6227c22
  czhu-cohere authored Dec 08, 2025
```
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
```
  f6227c22
08 Dec, 2025 1 commit

[DeepSeek v3.2] Make top-k work for any logit values. (#27568) · 184076c3

Daniel Cámpora authored Dec 08, 2025


Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

184076c3

07 Dec, 2025 2 commits

[Performance] Fused blockwise quant RMS norm (#27883) · af0444bf

ElizaWszola authored Dec 07, 2025


Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>

af0444bf

[Perf] Deepgemm fused layout kernel for activations, 4.3% throughput... · 541a2ef8

Wentao Ye authored Dec 07, 2025


[Perf] Deepgemm fused layout kernel for activations, 4.3% throughput improvement, 10.7% TTFT improvement. (#29546)
Signed-off-by: yewentao256 <zhyanwentao@126.com>

541a2ef8

29 Nov, 2025 1 commit

[Kernel][Quantization] add w4a8 support for marlin kernel (#24722) · 1656ad37

Jinzhen Lin authored Nov 29, 2025


Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>

1656ad37

26 Nov, 2025 1 commit
- [Performance][MLA][ROCm] Remove redundant D2D copy in deepseek (#27457) · d9d342d2
  Pleaplusone authored Nov 26, 2025
```
Signed-off-by: ganyi <ygan@amd.com>
```
  d9d342d2
25 Nov, 2025 1 commit

[Perf][Deepseek] optimize gather_and_maybe_dequant_cache kernel's perf for... · 77e10c9c

Pleaplusone authored Nov 25, 2025


[Perf][Deepseek] optimize gather_and_maybe_dequant_cache kernel's perf for extremely long sequence (#28029)
Signed-off-by: ganyi <ygan@amd.com>

77e10c9c

20 Nov, 2025 1 commit

cleanup at::Tag::needs_fixed_stride_order (#28974) · a903d59f

Boyuan Feng authored Nov 20, 2025


Signed-off-by: Boyuan Feng <boyuan@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

a903d59f

12 Nov, 2025 1 commit
- [ROCm] [Bugfix] Fix `fused_qknorm_rope_kernel` rocm compatibility (#28500) · edb59a94
  TJian authored Nov 12, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  edb59a94
11 Nov, 2025 1 commit
- [Kernel][Perf] fuse QK Norm and RoPE into one cuda kernel for Qwen Model (#27165) · 68c09efc
  zhrrr authored Nov 12, 2025
```
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
```
  68c09efc
02 Nov, 2025 1 commit
- [V1] [Hybrid] Mamba1 Automatic Prefix Caching (#26377) · 00b31a36
  Asaf Joseph Gardin authored Nov 02, 2025
```
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
```
  00b31a36
24 Oct, 2025 1 commit
- [Kernel] Add GPTQv2 format support for low-bit or asymmetric quantization, by... · 5cc6bddb
  Xiangyu Li authored Oct 24, 2025
```
[Kernel] Add GPTQv2 format support for low-bit or asymmetric quantization, by adapting gptq_gemm (#26092)
```
  5cc6bddb
21 Oct, 2025 2 commits

[Deepseek v3.2] Remove extra logics in indexer (#26465) · 09a7e6f6

Lain authored Oct 21, 2025


Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Signed-off-by: Lain <siyuanf@nvidia.com>
Co-authored-by: Daniel Campora <961215+dcampora@users.noreply.github.com>

09a7e6f6

[Deepseek v3.2] Optimize top_k_per_row (#26763) · 80e94529
Daniel Cámpora authored Oct 21, 2025
```
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
```
80e94529

17 Oct, 2025 1 commit
- [Chore] Remove unused `PolyNorm` layer (#27110) · 3125d799
  Isotr0py authored Oct 18, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  3125d799
10 Oct, 2025 1 commit

Silu v2 (#25074) · 7b03584d

Elvir Crnčević authored Oct 10, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: elvircrn <elvircrn@gmail.com>
Signed-off-by: Elvir Crnčević <elvircrn@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>

7b03584d

08 Oct, 2025 1 commit

Add gather_indexer_k_quant_cache kernel (#25931) · 127c8b78

Barry Kang authored Oct 08, 2025


Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Simon Mo <simon.mo@hey.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>

127c8b78

07 Oct, 2025 1 commit

Add topk logits torch op for DS3.2. (#25945) · e1098ced

Daniel Cámpora authored Oct 07, 2025

Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Signed-off-by: Daniel Cámpora <961215+dcampora@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

e1098ced

03 Oct, 2025 1 commit
- [CI/Build] Conditionally register cutlass_fp4_group_mm to fix building on Hopper (#26138) · 5d5146ee
  Michael Goin authored Oct 02, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  5d5146ee
30 Sep, 2025 1 commit

[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) · fa7e254a

Yongye Zhu authored Sep 30, 2025


Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

fa7e254a