Commits · b1235c3e109746abcf638ed9bf76d43f03489b5e · OpenDAS / vllm_cscc

12 Jul, 2025 2 commits
- [Bugfix] Lazy import fused_experts in BitsAndBytesMoEMethod to avoid break... · b1235c3e
  Li, Jiang authored Jul 12, 2025
```
[Bugfix] Lazy import fused_experts in BitsAndBytesMoEMethod to avoid break not-cuda-alike devices  (#20822)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  b1235c3e
- [Misc] Restrict deep_gemm's log output (#20827) · 44d02f54
  Jee Jee Li authored Jul 12, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  44d02f54
11 Jul, 2025 6 commits
- [Misc] Add unit tests for MoE ModularKernel combinations + Profiling utility (#20449) · 53fa4573
  Varun Sundar Rabindranath authored Jul 11, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  53fa4573
- [Quantization][1/N] MoE support BNB-Inflight Quantization (#20061) · 8020e98c
  Jee Jee Li authored Jul 11, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  8020e98c
- [V1] Enable Mamba2 layers other than MambaMixer2 in the v1 engine (#20660) · 5d09152f
  nopperl authored Jul 11, 2025
```
Signed-off-by: nopperl <54780682+nopperl@users.noreply.github.com>
```
  5d09152f
- [Perf][fp8] Use CustomOp abstraction for fp8 quant for better perf (#19830) · 31d5c179
  Luka Govedič authored Jul 11, 2025
```
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  31d5c179
- [Feature] Integrate SM100 DeepGEMM support (#20087) · e2de455c
  Wentao Ye authored Jul 10, 2025
  
  e2de455c
- [fix]: disable cutlass block scaled group gemm for EP (#20781) · 5923ab95
  Duncan Moss authored Jul 10, 2025
```
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
```
  5923ab95
10 Jul, 2025 4 commits

[Misc] MoE ModularKernel : Introduce TopKWeightAndReduce (#20648) · f0c98cae

Varun Sundar Rabindranath authored Jul 10, 2025


Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>

f0c98cae

[Bugfix] Fused MoE Modular Kernel chunking loop (#20392) · fdadb6f4

Varun Sundar Rabindranath authored Jul 10, 2025


Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>

fdadb6f4

[Bugfix] Remove assertion of expert_map being None (#20714) · 3de2ed76
Ming Yang authored Jul 10, 2025
```
Signed-off-by: Ming Yang <yming@meta.com>
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
3de2ed76

[Misc] DP : Add ExpertTokensMetadata (#20332) · 805d62ca

Varun Sundar Rabindranath authored Jul 09, 2025


Signed-off-by: Varun <vsundarr@redhat.com>
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>

805d62ca

09 Jul, 2025 5 commits
- [Feature][Quantization] MXFP4 support for MOE models (#17888) · 332d4cb1
  fxmarty-amd authored Jul 09, 2025
```
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: Bowen Bao <bowenbao@amd.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Bowen Bao <bowenbao@amd.com>
```
  332d4cb1
- [Kernel] Add Conch backend for mixed-precision linear layer (#19818) · bf03ff35
  Jacob Manning authored Jul 09, 2025
```
Signed-off-by: Jacob Manning <jmanning+oss@stackav.com>
```
  bf03ff35
- [Kernel] Triton implementation of causal-conv1d for Mamba-based models (#18218) · 47043eb6
  Tuan, Hoang-Trong authored Jul 09, 2025
```
Signed-off-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
Co-authored-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  47043eb6
- Support Llama 4 for cutlass_moe_fp4 (#20453) · 31b96d1c
  Michael Goin authored Jul 10, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  31b96d1c
- [feat] enable SM100 CUTLASS block scaled group gemm for smaller batch sizes (#20640) · 97abeb1d
  Duncan Moss authored Jul 08, 2025
```
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
```
  97abeb1d
08 Jul, 2025 2 commits
- [Bugfix] Fix topk_ids indices_type for CUTLASS w8a8 FP8 MoE (#20166) · c438183e
  Ming Yang authored Jul 08, 2025
```
Signed-off-by: Ming Yang <yming@meta.com>
```
  c438183e
- [Bugfix] Fix Maverick correctness by filling zero to cache space in cutlass_moe (#20167) · afb7cff1
  Ming Yang authored Jul 07, 2025
```
Signed-off-by: Ming Yang <yming@meta.com>
```
  afb7cff1
06 Jul, 2025 4 commits
- [Bugfix] Add `use_cross_encoder` flag to use correct activation in `ClassifierPooler` (#20527) · c18b3b8e
  Cyrus Leung authored Jul 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c18b3b8e
- [Benchmark] Add support for multiple batch size benchmark through CLI in... · cede942b
  Brayden Zhong authored Jul 06, 2025
```
[Benchmark] Add support for multiple batch size benchmark through CLI in `benchmark_moe.py` (#20516)
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  cede942b
- [TPU][Bugfix] fix the MoE OOM issue (#20339) · 4548c03c
  Chengji Yao authored Jul 05, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  4548c03c
- [Bugfix] Fix missing per_act_token parameter in compressed_tensors_moe (#20509) · 43287082
  Lucia Fang authored Jul 06, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  43287082
05 Jul, 2025 1 commit
- Fix unknown attribute of topk_indices_dtype in CompressedTensorsW8A8Fp8MoECutlassMethod (#20507) · 8aeaa910
  Lucia Fang authored Jul 05, 2025
```
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
```
  8aeaa910
04 Jul, 2025 4 commits

[feat]: CUTLASS block scaled group gemm for SM100 (#19757) · 3d184b95
Duncan Moss authored Jul 04, 2025
```
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Co-authored-by: Duncan Moss <dmoss@nvidia.com>
```
3d184b95

Enable V1 for Hybrid SSM/Attention Models (#20016) · 2f35a022

Thomas Parnell authored Jul 04, 2025


Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>

2f35a022

Support Llama 4 for fused_marlin_moe (#20457) · 0e3fe896
Michael Goin authored Jul 04, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
0e3fe896
[Misc] Add SPDX-FileCopyrightText (#20428) · 1caca5a5
Jee Jee Li authored Jul 04, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
1caca5a5

03 Jul, 2025 5 commits
- [Kernel] Enable fp8 support for pplx and BatchedTritonExperts. (#18864) · 78fe7753
  bnellnm authored Jul 03, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  78fe7753
- [Model][2/N] Automatic conversion of CrossEncoding model (#19978) · 6f1229f9
  wang.yuqi authored Jul 03, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  6f1229f9
- [Quantization] Bump to use latest bitsandbytes (#20424) · 1819fbda
  Jee Jee Li authored Jul 03, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1819fbda
- [Bugfix][CI/CD][CPU] Fix CPU CI tests (#20383) · 0ec3779d
  Li, Jiang authored Jul 03, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  0ec3779d
- [Bugfix] Fix import of CutlassExpertsFp8 in compressed_tensors_moe.py (#20381) · 2e25bb12
  bnellnm authored Jul 02, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  2e25bb12
02 Jul, 2025 5 commits
- [Kernels] MoE refactor (#19636) · c1909e7e
  bnellnm authored Jul 02, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
```
  c1909e7e
- [Bugfix] Fix dynamic rotary embedding (#20343) · 1a03dd49
  Cyrus Leung authored Jul 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1a03dd49
- [Refactor] Remove Unused Env `VLLM_ENABLE_MOE_ALIGN_BLOCK_SIZE_TRITON` (#20334) · 9dae7d46
  Wentao Ye authored Jul 01, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  9dae7d46
- [Refactor] Remove duplicate `find_free_port` (#20333) · 7058d7dd
  Wentao Ye authored Jul 01, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  7058d7dd
- Enable group size 64 for Machete (#20290) · 3abfe221
  czhu-cohere authored Jul 01, 2025
```
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
```
  3abfe221
01 Jul, 2025 2 commits

[V1] [ROCm] Enable EP with AITER Fused MoE (#20270) · 02cabff2
TJian authored Jul 01, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
02cabff2

[Model]Add Tencent HunYuanMoEV1 Model Support (#20114) · ecad851c

aiyiwang2025 authored Jul 01, 2025


Signed-off-by: aiyiwang <aiyiwang@tencent.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: quinnrong <quinnrong@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

ecad851c