---
id: pr-flashinfer-2951
repo: flashinfer-ai/flashinfer
pr: 2951
title: 'feat: Add DCP All-to-All kernel for context-parallel attention reduction'
author: davidjpyu
date: '2026-04-23'
url: https://github.com/flashinfer-ai/flashinfer/pull/2951
source_category: upstream-code
architectures:
- sm100
- sm90
tags:
- attention_kv
- benchmark
- benchmark_test
- blackwell
- compiler_runtime
- cuda kernel
- hopper
- moe_routing
- sm100
- sm90
techniques:
- cache-policy
hardware_features:
- blackwell
- hopper
kernel_types:
- attention
- compiler
- decode
- moe
- reduction
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 498e8378827973065bb25a622b15420ff57509f6
inclusion_reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.
changed_paths:
- benchmarks/bench_dcp_alltoall.py
- csrc/nv_internal/tensorrt_llm/kernels/cudaAsyncOps.cuh
- csrc/nv_internal/tensorrt_llm/kernels/helixAllToAll.cu
- csrc/nv_internal/tensorrt_llm/kernels/helixAllToAll.h
- csrc/nv_internal/tensorrt_llm/kernels/ll128Proto.cuh
- csrc/nv_internal/tensorrt_llm/kernels/moeCommKernelsCommon.h
- csrc/trtllm_dcp_alltoall.cu
- flashinfer/aot.py
- flashinfer/comm/__init__.py
- flashinfer/comm/dcp_alltoall.py
- flashinfer/jit/__init__.py
- flashinfer/jit/comm.py
- tests/comm/test_dcp_alltoall.py
- tests/comm/test_mnnvl_dcp_alltoall.py
artifact_dir: evidence/pull-bundles/flashinfer/gh-2951
head_ref_oid: 2c89bdfece444e502f35d77e99c148a743d4271f
---
# feat: Add DCP All-to-All kernel for context-parallel attention reduction

- Repository: `flashinfer-ai/flashinfer`
- PR: [2951](https://github.com/flashinfer-ai/flashinfer/pull/2951)
- Decision: `Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.`
- Reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.

## Changed Paths

- `benchmarks/bench_dcp_alltoall.py`
- `csrc/nv_internal/tensorrt_llm/kernels/cudaAsyncOps.cuh`
- `csrc/nv_internal/tensorrt_llm/kernels/helixAllToAll.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/helixAllToAll.h`
- `csrc/nv_internal/tensorrt_llm/kernels/ll128Proto.cuh`
- `csrc/nv_internal/tensorrt_llm/kernels/moeCommKernelsCommon.h`
- `csrc/trtllm_dcp_alltoall.cu`
- `flashinfer/aot.py`
- `flashinfer/comm/__init__.py`
- `flashinfer/comm/dcp_alltoall.py`
- `flashinfer/jit/__init__.py`
- `flashinfer/jit/comm.py`
- `tests/comm/test_dcp_alltoall.py`
- `tests/comm/test_mnnvl_dcp_alltoall.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-2951`
