---
id: pr-vllm-15956
repo: vllm-project/vllm
pr: 15956
title: Modularize fused experts and integrate PPLX kernels
author: bnellnm
date: '2025-05-14'
url: https://github.com/vllm-project/vllm/pull/15956
source_category: upstream-code
architectures:
- sm100
tags:
- attention
- fp8
- gemm
- kernel-fusion
- mla
- moe
- quantization
- tma
techniques:
- kernel-fusion
hardware_features:
- fp8
- tma
kernel_types:
- attention
- gemm
- mla
- moe
- quantization
languages:
- cuda-cpp
- triton
captured_at: '2026-04-27'
status: merged
merge_sha: f9c069c85e029830094ff9abb926ffbf37b7c7e7
inclusion_reason: kernel file changes
changed_paths:
- csrc/activation_kernels.cu
- csrc/dispatch_utils.h
- csrc/moe/moe_align_sum_kernels.cu
- csrc/moe/topk_softmax_kernels.cu
- examples/offline_inference/data_parallel.py
- tests/kernels/moe/test_batched_moe.py
- tests/kernels/moe/test_cutlass_moe.py
- tests/kernels/moe/test_moe.py
- tests/kernels/moe/test_pplx_moe.py
- tests/kernels/moe/test_triton_moe_ptpc_fp8.py
- tests/kernels/quantization/test_block_fp8.py
- tests/kernels/quantization/test_block_int8.py
- vllm/distributed/parallel_state.py
- vllm/distributed/utils.py
- vllm/forward_context.py
- vllm/model_executor/layers/fused_moe/__init__.py
- vllm/model_executor/layers/fused_moe/cutlass_moe.py
- vllm/model_executor/layers/fused_moe/deep_gemm_moe.py
- vllm/model_executor/layers/fused_moe/fused_batched_moe.py
- vllm/model_executor/layers/fused_moe/fused_moe.py
- vllm/model_executor/layers/fused_moe/layer.py
- vllm/model_executor/layers/fused_moe/modular_kernel.py
- vllm/model_executor/layers/fused_moe/moe_permute_unpermute.py
- vllm/model_executor/layers/fused_moe/pplx_prepare_finalize.py
- vllm/model_executor/layers/fused_moe/prepare_finalize.py
- vllm/model_executor/layers/fused_moe/triton_deep_gemm_moe.py
- vllm/model_executor/layers/fused_moe/utils.py
- vllm/model_executor/layers/quantization/fp8.py
- vllm/model_executor/models/dbrx.py
- vllm/model_executor/models/deepseek_v2.py
- vllm/model_executor/models/llama4.py
- vllm/model_executor/models/qwen2_moe.py
- vllm/model_executor/models/qwen3_moe.py
- vllm/platforms/cuda.py
- vllm/v1/attention/backends/mla/common.py
- vllm/v1/worker/gpu_worker.py
- vllm/v1/worker/tpu_worker.py
- vllm/worker/cpu_worker.py
- vllm/worker/hpu_worker.py
- vllm/worker/tpu_worker.py
- vllm/worker/worker.py
- vllm/worker/xpu_worker.py
head_ref_oid: 1f91cfd28eb3e190950a2a36bf53b2e912bb2b70
artifact_dir: evidence/pull-bundles/vllm/gh-15956
---
# Modularize fused experts and integrate PPLX kernels

- Repository: `vllm-project/vllm`
- PR: [15956](https://github.com/vllm-project/vllm/pull/15956)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `csrc/activation_kernels.cu`
- `csrc/dispatch_utils.h`
- `csrc/moe/moe_align_sum_kernels.cu`
- `csrc/moe/topk_softmax_kernels.cu`
- `examples/offline_inference/data_parallel.py`
- `tests/kernels/moe/test_batched_moe.py`
- `tests/kernels/moe/test_cutlass_moe.py`
- `tests/kernels/moe/test_moe.py`
- `tests/kernels/moe/test_pplx_moe.py`
- `tests/kernels/moe/test_triton_moe_ptpc_fp8.py`
- `tests/kernels/quantization/test_block_fp8.py`
- `tests/kernels/quantization/test_block_int8.py`
- `vllm/distributed/parallel_state.py`
- `vllm/distributed/utils.py`
- `vllm/forward_context.py`
- `vllm/model_executor/layers/fused_moe/__init__.py`
- `vllm/model_executor/layers/fused_moe/cutlass_moe.py`
- `vllm/model_executor/layers/fused_moe/deep_gemm_moe.py`
- `vllm/model_executor/layers/fused_moe/fused_batched_moe.py`
- `vllm/model_executor/layers/fused_moe/fused_moe.py`
- `vllm/model_executor/layers/fused_moe/layer.py`
- `vllm/model_executor/layers/fused_moe/modular_kernel.py`
- `vllm/model_executor/layers/fused_moe/moe_permute_unpermute.py`
- `vllm/model_executor/layers/fused_moe/pplx_prepare_finalize.py`
- `vllm/model_executor/layers/fused_moe/prepare_finalize.py`
- `vllm/model_executor/layers/fused_moe/triton_deep_gemm_moe.py`
- `vllm/model_executor/layers/fused_moe/utils.py`
- `vllm/model_executor/layers/quantization/fp8.py`
- `vllm/model_executor/models/dbrx.py`
- `vllm/model_executor/models/deepseek_v2.py`
- `vllm/model_executor/models/llama4.py`
- `vllm/model_executor/models/qwen2_moe.py`
- `vllm/model_executor/models/qwen3_moe.py`
- `vllm/platforms/cuda.py`
- `vllm/v1/attention/backends/mla/common.py`
- `vllm/v1/worker/gpu_worker.py`
- `vllm/v1/worker/tpu_worker.py`
- `vllm/worker/cpu_worker.py`
- `vllm/worker/hpu_worker.py`
- `vllm/worker/tpu_worker.py`
- `vllm/worker/worker.py`
- `vllm/worker/xpu_worker.py`

## Evidence Bundle

`evidence/pull-bundles/vllm/gh-15956`
