Commits · 5b31cb1781e594aae29d878b1acde3e2f900bc41 · OpenDAS / vllm_cscc

29 Aug, 2025 1 commit
- Tuned H100/H200 triton fp8 block configs for fused_qkv_a_proj (#23939) · b7adf94c
  Michael Goin authored Aug 29, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b7adf94c
28 Aug, 2025 6 commits
- [NVIDIA] Support SiluMul + NVFP4 quant fusion (#23671) · 16a45b3a
  elvischenv authored Aug 29, 2025
```
Signed-off-by: jindih <jindih@nvidia.com>
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: jindih <jindih@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedic <lgovedic@redhat.com>
```
  16a45b3a
- [Model][gpt-oss] Support DP+EP for GPT-OSS with FlashInfer trtllm-gen MoE (#23819) · 95089607
  Po-Han Huang (NVIDIA) authored Aug 28, 2025
```
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
```
  95089607
- [Doc]: fix typos in Python scripts (#23828) · d3da2eea
  Didier Durand authored Aug 28, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  d3da2eea
- [FIXBUG] Add return_success parameter to moe_wna16_weight_loader function (#22797) · 3462c1c5
  JartX authored Aug 28, 2025
```
Signed-off-by: JartX <sagformas@epdcenter.es>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  3462c1c5
- [Transform] [Quantization] Add transforms to compressed tensors (#22486) · 22feac8e
  Kyle Sayers authored Aug 28, 2025
  
  22feac8e
- [Perf] Tune configs for triton block fp8 gemm H100/H200 (#23748) · a781e84e
  Michael Goin authored Aug 27, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a781e84e
27 Aug, 2025 6 commits
- [Bugfix] Fix Marlin NVFP4 for modelopt (#23659) · f9ca2b40
  Michael Goin authored Aug 27, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f9ca2b40
- DP/EP Support for gpt-oss with deepep-ht comm kernel on SM100 (#23608) · 082cc07e
  Yongye Zhu authored Aug 27, 2025
  
  082cc07e
- [Feature] Add Hopper DeepGEMM E8M0 for DeepSeekV3.1 scale_fmt (#23666) · 3af47c3c
  Wentao Ye authored Aug 27, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  3af47c3c
- [Quantization] Expand compressed-tensors MoE matching logic to support NFP4 + FP8 MoEs (#22674) · d272415e
  Dipika Sikka authored Aug 27, 2025
```
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
```
  d272415e
- [Bugfix] Lazy import gpt_oss_triton_kernels_moe for mxfp4 (#23678) · de02b07d
  Michael Goin authored Aug 26, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  de02b07d
- [quantization] use channel scales for w4a8 + misc fixes (#23570) · 2c2b140a
  czhu-cohere authored Aug 26, 2025
```
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
```
  2c2b140a
26 Aug, 2025 3 commits

[Kernel] Added flashinfer fp8 per-tensor gemms (#22895) · f66673a3
nvjullin authored Aug 26, 2025
```
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
f66673a3
[Perf] Remove duplicated NVFP4 blockscales to save memory (#23379) · d52358c1
Michael Goin authored Aug 26, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
d52358c1

Update Flashinfer to 0.2.14.post1 (#23537) · ae067888

weiliang authored Aug 26, 2025


Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: siyuanf <siyuanf@nvidia.com>
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

ae067888

25 Aug, 2025 1 commit
- [Perf] Add Triton config for DeepSeek V3 FP8 EP32 H200 (#23504) · 504d9143
  Ming Yang authored Aug 24, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  504d9143
24 Aug, 2025 1 commit
- [kernel] Support W4A8 on Hopper (#23198) · e76e2335
  czhu-cohere authored Aug 24, 2025
```
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
```
  e76e2335
23 Aug, 2025 1 commit

Quantization: support FP4 quantized models on AMD CDNA2/CDNA3 GPUs (#22527) · fa78de9d

Daifeng Li authored Aug 23, 2025


Signed-off-by: feng <fengli1702@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

fa78de9d

22 Aug, 2025 4 commits
- [NVIDIA][torch.compile] Support Flashinfer TRTLLM FP8-q/kv NVFP4-out Attention Kernel (#22703) · 24d0c9e6
  elvischenv authored Aug 23, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  24d0c9e6
- [BugFix][AMD][Quantization] Fix torch.compile issue where wvSplitKQ not being... · cc7ae5e7
  rasmith authored Aug 22, 2025
```
[BugFix][AMD][Quantization] Fix torch.compile issue where wvSplitKQ not being called when it should when using quantized FP8 model (#22281)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  cc7ae5e7
- [Quantization] Allow GGUF quantization to skip unquantized layer (#23188) · 4645024d
  Isotr0py authored Aug 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  4645024d
- [Feature] Enable DeepGEMM Linear on B200; 1.5% E2E throughput improvement (#23351) · 394591e3
  Wentao Ye authored Aug 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  394591e3
21 Aug, 2025 3 commits

[Bug] Fix R1 Accuracy 0 Bug (#23294) · 48bfb0c9

Wentao Ye authored Aug 21, 2025


Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

48bfb0c9

Fix nvfp4 swizzling (#23140) · 0278f1ac

Yi Liu authored Aug 22, 2025


Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

0278f1ac

[CPU] Refactor CPU W8A8 scaled_mm (#23071) · 7be5d113
Li, Jiang authored Aug 21, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
7be5d113

20 Aug, 2025 2 commits
- [Kernel/Quant] Remove the original marlin format and qqq (#23204) · 0cdbf5e6
  Michael Goin authored Aug 20, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  0cdbf5e6
- [Kernel] CUTLASS MoE FP8: Integrate cuda moe permute/unpermute (#23045) · b17109be
  shixianc authored Aug 20, 2025
```
Signed-off-by: Shixian Cui <shixian@amazon.com>
```
  b17109be
19 Aug, 2025 2 commits
- [NVIDIA] Add SM100 Flashinfer Cutlass MoE fp8 backend (#22357) · a38b8af4
  amirkl94 authored Aug 20, 2025
```
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
```
  a38b8af4
- [Bugfix] Fix accuracy issue when using flashinfer cutlass moe, TP=1 and modelopt. (#23125) · b94faf9d
  bnellnm authored Aug 19, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  b94faf9d
18 Aug, 2025 1 commit
- Use Blackwell FlashInfer MXFP4 MoE by default if available (#23008) · 6d25e3fd
  Michael Goin authored Aug 18, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6d25e3fd
17 Aug, 2025 3 commits
- fix: gptq marlin weight loading failure (#23066) · 0fc8fa75
  Simon Mo authored Aug 17, 2025
  
  0fc8fa75
- [Kernel] Add cuda kernel for gpt_oss activation (#22951) · 4d4061b6
  Jee Jee Li authored Aug 17, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  4d4061b6
- [Bugfix] fix qwen3 moe fp8 accuracy issue (#23031) · a258ad8b
  Jinzhen Lin authored Aug 17, 2025
```
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
```
  a258ad8b
16 Aug, 2025 1 commit

[Kernel/Quant] Remove AQLM (#22943) · 4fc722ec

Michael Goin authored Aug 16, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

4fc722ec

15 Aug, 2025 5 commits
- [Kernels] Clean up FusedMoeMethodBase and modular kernel setup. Remove extra... · 8ad7285e
  bnellnm authored Aug 15, 2025
```
[Kernels] Clean up FusedMoeMethodBase and modular kernel setup.  Remove extra arguments from modular kernel methods. (#22035)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  8ad7285e
- [BugFix] Skip the Q component for QKVParallelLinear in the case of... · 6b04039a
  sstamenk authored Aug 15, 2025
```
[BugFix] Skip the Q component for QKVParallelLinear in the case of QKVCrossParallelLinear since its width is 0 (#22369)
Signed-off-by: sstamenk <sstamenk@amd.com>
```
  6b04039a
- refactor: Change scaling factors calculation for flashinfer FusedMoE (#22812) · b4cef5e6
  amirkl94 authored Aug 15, 2025
```
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  b4cef5e6
- Revert "[Kernel] Add cuda kernel for gpt_oss activation" (#22948) · f1f0d2fa
  Simon Mo authored Aug 14, 2025
  
  f1f0d2fa
- [Kernel] Add cuda kernel for gpt_oss activation (#22538) · 81f4b964
  Jee Jee Li authored Aug 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  81f4b964