Commits · 7ae75fa6d02afc45637b060ce7a535a1bd547afd · OpenDAS / vllm_cscc

26 Jul, 2025 2 commits
- [Feature] Add support for MoE models in the calibration-free RTN-based quantization (#20766) · 7ae75fa6
  Alex Kogan authored Jul 25, 2025
```
Signed-off-by: Alex Kogan <alex.kogan@oracle.com>
```
  7ae75fa6
- [Perf] Cuda Kernel for Int8 Per Token Group Quant (#21476) · 75d29cf4
  Wentao Ye authored Jul 25, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  75d29cf4
25 Jul, 2025 1 commit
- [Bug] Fix DeepGemm Init Error (#21554) · 633f6e80
  Wentao Ye authored Jul 24, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  633f6e80
24 Jul, 2025 1 commit
- Update flashinfer CUTLASS MoE Kernel (#21408) · 1b25f1fe
  Shu Wang authored Jul 24, 2025
```
Signed-off-by: Shu Wang. <shuw@nvidia.com>
```
  1b25f1fe
22 Jul, 2025 4 commits
- [Perf] Cuda Kernel for Per Token Group Quant (#21083) · 774d0c01
  Wentao Ye authored Jul 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  774d0c01
- [feat]: add SM100 support for cutlass FP8 groupGEMM (#20447) · 2c8db17c
  Duncan Moss authored Jul 22, 2025
```
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  2c8db17c
- [perf] Add fused MLA QKV + strided layernorm (#21116) · 4fb56914
  Mickaël Seznec authored Jul 22, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  4fb56914
- Revert "[Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) (#21334) · e7b20426
  Ming Yang authored Jul 21, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  e7b20426
21 Jul, 2025 1 commit
- Add Nvidia ModelOpt config adaptation (#19815) · 6b46c4b6
  Zhiyu authored Jul 21, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
  6b46c4b6
19 Jul, 2025 1 commit

[NVIDIA] Add SM100 Flashinfer MoE blockscale fp8 backend for low latency (#20645) · 6d0734c5

Kaixi Hou authored Jul 19, 2025


Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

6d0734c5

18 Jul, 2025 1 commit

[Core] FlashInfer CUTLASS fused MoE backend (NVFP4) (#20037) · c7d8724e

Shu Wang authored Jul 17, 2025


Signed-off-by: shuw <shuw@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

c7d8724e

17 Jul, 2025 2 commits
- [Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) · 9fb2d220
  ElizaWszola authored Jul 17, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
  9fb2d220
- [Bugfix] Fix Machete zero point issue for GPTQ models on SM90 (#21066) · 28a6d542
  Michael Goin authored Jul 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  28a6d542
16 Jul, 2025 3 commits

Support FP8 Quantization and Inference Run on Intel Gaudi (HPU) using INC... · 01513a33

Nir David authored Jul 16, 2025


Support FP8 Quantization and Inference Run on Intel Gaudi (HPU) using INC (Intel Neural Compressor) (#12010)
Signed-off-by: Nir David <ndavid@habana.ai>
Signed-off-by: Uri Livne <ulivne@habana.ai>
Co-authored-by: Uri Livne <ulivne@habana.ai>

01513a33

[Doc] Remove duplicate docstring (#21012) · 76ddeff2
Wentao Ye authored Jul 15, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
76ddeff2
[Bugfix] Correct per_act_token in CompressedTensorsW8A8Fp8MoECutlassM… (#20937) · fcb9f879
Ming Yang authored Jul 15, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
fcb9f879

15 Jul, 2025 3 commits
- Enabled BnB NF4 inference on Gaudi (#20172) · 016b8d1b
  Ruheena Suhani Shaik authored Jul 15, 2025
```
Signed-off-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
```
  016b8d1b
- Use w8a8 quantized matmul Pallas kernel (#19170) · d4170fad
  XiongfeiWei authored Jul 14, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  d4170fad
- [Bugfix] Fix incorrect dispatch for CutlassBlockScaledGroupedGemm and DeepGEMM (#20933) · bcdfb2a3
  Michael Goin authored Jul 15, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  bcdfb2a3
13 Jul, 2025 3 commits
- [V1] [ROCm] [AITER] Upgrade AITER to commit `916bf3c` and bugfix APIs (#20880) · 80d38b8a
  TJian authored Jul 13, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  80d38b8a
- [Bugfix] Fix a couple PPLX+CUTLASS MoE bugs (#20825) · 3b3b778d
  ElizaWszola authored Jul 13, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
  3b3b778d
- [Perf] Use Triton instead of Torch for DeepGEMM Per Token Group Quant (#20841) · 42d440c2
  Wentao Ye authored Jul 12, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  42d440c2
12 Jul, 2025 3 commits
- [Bugfix] Restrict Machete to only run on Hopper (#20830) · 6e2c176e
  Michael Goin authored Jul 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6e2c176e
- Enable ModelOpt Llama4 fp8 checkpoint deployment (#20419) · 4afe687a
  Zhiyu authored Jul 11, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
  4afe687a
- [Bugfix] Lazy import fused_experts in BitsAndBytesMoEMethod to avoid break... · b1235c3e
  Li, Jiang authored Jul 12, 2025
```
[Bugfix] Lazy import fused_experts in BitsAndBytesMoEMethod to avoid break not-cuda-alike devices  (#20822)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  b1235c3e
11 Jul, 2025 3 commits
- [Quantization][1/N] MoE support BNB-Inflight Quantization (#20061) · 8020e98c
  Jee Jee Li authored Jul 11, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  8020e98c
- [Perf][fp8] Use CustomOp abstraction for fp8 quant for better perf (#19830) · 31d5c179
  Luka Govedič authored Jul 11, 2025
```
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  31d5c179
- [Feature] Integrate SM100 DeepGEMM support (#20087) · e2de455c
  Wentao Ye authored Jul 10, 2025
  
  e2de455c
09 Jul, 2025 3 commits
- [Feature][Quantization] MXFP4 support for MOE models (#17888) · 332d4cb1
  fxmarty-amd authored Jul 09, 2025
```
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: Bowen Bao <bowenbao@amd.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Bowen Bao <bowenbao@amd.com>
```
  332d4cb1
- [Kernel] Add Conch backend for mixed-precision linear layer (#19818) · bf03ff35
  Jacob Manning authored Jul 09, 2025
```
Signed-off-by: Jacob Manning <jmanning+oss@stackav.com>
```
  bf03ff35
- Support Llama 4 for cutlass_moe_fp4 (#20453) · 31b96d1c
  Michael Goin authored Jul 10, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  31b96d1c
08 Jul, 2025 1 commit
- [Bugfix] Fix topk_ids indices_type for CUTLASS w8a8 FP8 MoE (#20166) · c438183e
  Ming Yang authored Jul 08, 2025
```
Signed-off-by: Ming Yang <yming@meta.com>
```
  c438183e
05 Jul, 2025 1 commit
- Fix unknown attribute of topk_indices_dtype in CompressedTensorsW8A8Fp8MoECutlassMethod (#20507) · 8aeaa910
  Lucia Fang authored Jul 05, 2025
```
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
```
  8aeaa910
04 Jul, 2025 3 commits
- [feat]: CUTLASS block scaled group gemm for SM100 (#19757) · 3d184b95
  Duncan Moss authored Jul 04, 2025
```
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Co-authored-by: Duncan Moss <dmoss@nvidia.com>
```
  3d184b95
- Support Llama 4 for fused_marlin_moe (#20457) · 0e3fe896
  Michael Goin authored Jul 04, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  0e3fe896
- [Misc] Add SPDX-FileCopyrightText (#20428) · 1caca5a5
  Jee Jee Li authored Jul 04, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1caca5a5
03 Jul, 2025 4 commits
- [Kernel] Enable fp8 support for pplx and BatchedTritonExperts. (#18864) · 78fe7753
  bnellnm authored Jul 03, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  78fe7753
- [Quantization] Bump to use latest bitsandbytes (#20424) · 1819fbda
  Jee Jee Li authored Jul 03, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1819fbda
- [Bugfix][CI/CD][CPU] Fix CPU CI tests (#20383) · 0ec3779d
  Li, Jiang authored Jul 03, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  0ec3779d
- [Bugfix] Fix import of CutlassExpertsFp8 in compressed_tensors_moe.py (#20381) · 2e25bb12
  bnellnm authored Jul 02, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  2e25bb12