---
id: pr-cutlass-1795
repo: NVIDIA/cutlass
pr: 1795
title: Support for TMA Epilogue for Group Gemm and add pingpong ptr array & Group Gemm
author: Junkai-Wu
date: '2024-09-11'
url: https://github.com/NVIDIA/cutlass/pull/1795
source_category: upstream-code
architectures:
- sm90
tags:
- arch_pipeline
- epilogue
- gemm_quant
- norm_elementwise
- tma
techniques:
- async-copy
- epilogue-fusion
- fine-grained-quantization
- kernel-fusion
- pipeline-stages
- warp-specialization
hardware_features:
- hopper
- tma
kernel_types:
- gemm
- norm
- runtime
languages:
- cuda-cpp
- cute-dsl
captured_at: '2026-05-16'
status: merged
merge_sha: dbdae514e03f83968f8b7dd4fb064071b9bfbdd1
inclusion_reason: 'Extract the architecture assumption first: SM target, TMA/WGMMA/tcgen path, cluster use, and pipeline stages.'
changed_paths:
- examples/56_hopper_ptr_array_batched_gemm/56_hopper_ptr_array_batched_gemm.cu
- examples/57_hopper_grouped_gemm/57_hopper_grouped_gemm.cu
- examples/57_hopper_grouped_gemm/CMakeLists.txt
- include/cute/util/type_traits.hpp
- include/cutlass/epilogue/collective/builders/sm90_builder.inl
- include/cutlass/epilogue/collective/detail.hpp
- include/cutlass/epilogue/collective/sm90_epilogue_array_tma_warpspecialized.hpp
- include/cutlass/epilogue/dispatch_policy.hpp
- include/cutlass/epilogue/fusion/operations.hpp
- include/cutlass/epilogue/fusion/sm90_callbacks_tma_warpspecialized.hpp
- include/cutlass/epilogue/fusion/sm90_visitor_load_tma_warpspecialized.hpp
- include/cutlass/gemm/collective/builders/sm90_gmma_builder.inl
- include/cutlass/gemm/collective/sm90_mma_array_tma_gmma_ss_warpspecialized.hpp
- include/cutlass/gemm/dispatch_policy.hpp
- include/cutlass/gemm/kernel/gemm_universal.hpp
- include/cutlass/gemm/kernel/sm90_gemm_array_tma_warpspecialized_cooperative.hpp
- include/cutlass/gemm/kernel/sm90_gemm_array_tma_warpspecialized_pingpong.hpp
- test/unit/gemm/device/CMakeLists.txt
- test/unit/gemm/device/gemm_testbed_3x_ptr_array.hpp
- test/unit/gemm/device/sm90_gemm_f16_f16_f16_tensor_op_f32_group_gemm.cu
- test/unit/gemm/device/sm90_gemm_f16_f16_f16_tensor_op_f32_group_gemm_pingpong.cu
- test/unit/gemm/device/sm90_gemm_f16_f16_f16_tensor_op_f32_ptr_array.cu
- test/unit/gemm/device/sm90_gemm_f16_f16_f16_tensor_op_f32_ptr_array_pingpong.cu
artifact_dir: evidence/pull-bundles/cutlass/gh-1795
head_ref_oid: 95b569c89384463ac6f5b133bbe14e481ea90a8b
---
# Support for TMA Epilogue for Group Gemm and add pingpong ptr array & Group Gemm

- Repository: `NVIDIA/cutlass`
- PR: [1795](https://github.com/NVIDIA/cutlass/pull/1795)
- Decision: `Extract the architecture assumption first: SM target, TMA/WGMMA/tcgen path, cluster use, and pipeline stages.`
- Reason: Extract the architecture assumption first: SM target, TMA/WGMMA/tcgen path, cluster use, and pipeline stages.

## Changed Paths

- `examples/56_hopper_ptr_array_batched_gemm/56_hopper_ptr_array_batched_gemm.cu`
- `examples/57_hopper_grouped_gemm/57_hopper_grouped_gemm.cu`
- `examples/57_hopper_grouped_gemm/CMakeLists.txt`
- `include/cute/util/type_traits.hpp`
- `include/cutlass/epilogue/collective/builders/sm90_builder.inl`
- `include/cutlass/epilogue/collective/detail.hpp`
- `include/cutlass/epilogue/collective/sm90_epilogue_array_tma_warpspecialized.hpp`
- `include/cutlass/epilogue/dispatch_policy.hpp`
- `include/cutlass/epilogue/fusion/operations.hpp`
- `include/cutlass/epilogue/fusion/sm90_callbacks_tma_warpspecialized.hpp`
- `include/cutlass/epilogue/fusion/sm90_visitor_load_tma_warpspecialized.hpp`
- `include/cutlass/gemm/collective/builders/sm90_gmma_builder.inl`
- `include/cutlass/gemm/collective/sm90_mma_array_tma_gmma_ss_warpspecialized.hpp`
- `include/cutlass/gemm/dispatch_policy.hpp`
- `include/cutlass/gemm/kernel/gemm_universal.hpp`
- `include/cutlass/gemm/kernel/sm90_gemm_array_tma_warpspecialized_cooperative.hpp`
- `include/cutlass/gemm/kernel/sm90_gemm_array_tma_warpspecialized_pingpong.hpp`
- `test/unit/gemm/device/CMakeLists.txt`
- `test/unit/gemm/device/gemm_testbed_3x_ptr_array.hpp`
- `test/unit/gemm/device/sm90_gemm_f16_f16_f16_tensor_op_f32_group_gemm.cu`
- `test/unit/gemm/device/sm90_gemm_f16_f16_f16_tensor_op_f32_group_gemm_pingpong.cu`
- `test/unit/gemm/device/sm90_gemm_f16_f16_f16_tensor_op_f32_ptr_array.cu`
- `test/unit/gemm/device/sm90_gemm_f16_f16_f16_tensor_op_f32_ptr_array_pingpong.cu`

## Evidence Bundle

`evidence/pull-bundles/cutlass/gh-1795`
