---
id: pr-sglang-3216
repo: sgl-project/sglang
pr: 3216
title: add tensorrt_llm common and cutlass_extensions as 3rdparty
author: zhyncs
date: '2025-01-30'
url: https://github.com/sgl-project/sglang/pull/3216
source_category: upstream-code
architectures:
- sm100
tags:
- epilogue-fusion
- fp8
- gemm
- kernel-fusion
- moe
- pipeline-stages
- quantization
- tma
techniques:
- epilogue-fusion
- kernel-fusion
- pipeline-stages
hardware_features:
- fp8
- tma
kernel_types:
- gemm
- moe
- quantization
languages:
- cuda-cpp
captured_at: '2026-04-27'
status: merged
merge_sha: 222ce6f1da31b6bfe168513ff85b2d5cad34fb85
inclusion_reason: kernel file changes
changed_paths:
- .clang-format-ignore
- sgl-kernel/3rdparty/tensorrt_llm/common/CMakeLists.txt
- sgl-kernel/3rdparty/tensorrt_llm/common/assert.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/cublasMMWrapper.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/cublasMMWrapper.h
- sgl-kernel/3rdparty/tensorrt_llm/common/cublasVersionCheck.h
- sgl-kernel/3rdparty/tensorrt_llm/common/cudaBf16Fallbacks.cuh
- sgl-kernel/3rdparty/tensorrt_llm/common/cudaDriverWrapper.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/cudaDriverWrapper.h
- sgl-kernel/3rdparty/tensorrt_llm/common/cudaFp8Utils.cu
- sgl-kernel/3rdparty/tensorrt_llm/common/cudaProfilerUtils.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/cudaTypeUtils.cuh
- sgl-kernel/3rdparty/tensorrt_llm/common/customAllReduceUtils.h
- sgl-kernel/3rdparty/tensorrt_llm/common/envUtils.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/envUtils.h
- sgl-kernel/3rdparty/tensorrt_llm/common/logger.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/mathUtils.h
- sgl-kernel/3rdparty/tensorrt_llm/common/memoryUtils.cu
- sgl-kernel/3rdparty/tensorrt_llm/common/memoryUtils.h
- sgl-kernel/3rdparty/tensorrt_llm/common/mpiUtils.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/nvtxUtils.h
- sgl-kernel/3rdparty/tensorrt_llm/common/opUtils.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/opUtils.h
- sgl-kernel/3rdparty/tensorrt_llm/common/quantTypeUtils.cuh
- sgl-kernel/3rdparty/tensorrt_llm/common/reduceKernelUtils.cuh
- sgl-kernel/3rdparty/tensorrt_llm/common/stlUtils.h
- sgl-kernel/3rdparty/tensorrt_llm/common/stringUtils.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/timestampUtils.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/timestampUtils.h
- sgl-kernel/3rdparty/tensorrt_llm/common/tllmException.cpp
- sgl-kernel/3rdparty/tensorrt_llm/common/workspace.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/arch/copy_red_global.hpp
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/arch/mma.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/compute_occupancy.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue/collective/epilogue_moe_finalize.hpp
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue/thread/fused_activations.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue/threadblock/epilogue_per_row_per_col_scale.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue/threadblock/epilogue_tensor_op_int32.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue_helpers.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/builders/sm90_gmma_builder_gated.inl
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_builder_gated.hpp
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_mma_gated.hpp
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_gated_tma_gmma_ss_warpspecialized.hpp
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_gated_tma_gmma_ss_warpspecialized_fp8.hpp
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/device/gemm_universal_base_compat.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/device/splitk_gemm_grouped.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/default_fpA_intB_traits.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/default_int8_traits.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/default_splitk_gemm_grouped.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/fpA_intB_gemm.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/fused_moe_kernel.cuh
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/fused_moe_kernel_routine.cuh
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/fused_moe_kernel_traits.cuh
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/gemm_moe_problem_visitor.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/gemm_universal_gated.hpp
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/gemm_with_epilogue_visitor.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/mixed_gemm_B_layout.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/moe_cute_util.cuh
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/moe_cutlass_kernel.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/moe_problem_visitor.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/sm90_gemm_gated_tma_warpspecialized_cooperative.hpp
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/sm90_gemm_gated_tma_warpspecialized_pingpong.hpp
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/splitk_gemm_grouped.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/default_dq_mma.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/default_dq_mma_multistage.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/default_dq_mma_pipelined.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/default_mma.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/default_mma_bf16.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_base.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_multistage.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_multistage_finegrained.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_multistage_percol.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_pipelined.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_pipelined_finegrained.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_pipelined_percol.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/warp/default_mma_tensor_op.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/warp/mma_tensorop_compute_B_with_f16.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/warp/mma_tensorop_dequantizer.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm_configs.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/interleaved_numeric_conversion.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/tile_interleaved_layout.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/transform/threadblock/fine_grained_scale_zero_iterator.h
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/util/gather_tensor.hpp
- sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/weight_only_quant_op.h
- sgl-kernel/THIRDPARTYNOTICES.txt
- sgl-kernel/setup.py
head_ref_oid: cc787294daea0d34cd0b55bb776bd4a53b831270
artifact_dir: evidence/pull-bundles/sglang/gh-3216
---
# add tensorrt_llm common and cutlass_extensions as 3rdparty

- Repository: `sgl-project/sglang`
- PR: [3216](https://github.com/sgl-project/sglang/pull/3216)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `.clang-format-ignore`
- `sgl-kernel/3rdparty/tensorrt_llm/common/CMakeLists.txt`
- `sgl-kernel/3rdparty/tensorrt_llm/common/assert.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/cublasMMWrapper.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/cublasMMWrapper.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/cublasVersionCheck.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/cudaBf16Fallbacks.cuh`
- `sgl-kernel/3rdparty/tensorrt_llm/common/cudaDriverWrapper.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/cudaDriverWrapper.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/cudaFp8Utils.cu`
- `sgl-kernel/3rdparty/tensorrt_llm/common/cudaProfilerUtils.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/cudaTypeUtils.cuh`
- `sgl-kernel/3rdparty/tensorrt_llm/common/customAllReduceUtils.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/envUtils.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/envUtils.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/logger.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/mathUtils.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/memoryUtils.cu`
- `sgl-kernel/3rdparty/tensorrt_llm/common/memoryUtils.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/mpiUtils.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/nvtxUtils.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/opUtils.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/opUtils.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/quantTypeUtils.cuh`
- `sgl-kernel/3rdparty/tensorrt_llm/common/reduceKernelUtils.cuh`
- `sgl-kernel/3rdparty/tensorrt_llm/common/stlUtils.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/stringUtils.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/timestampUtils.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/timestampUtils.h`
- `sgl-kernel/3rdparty/tensorrt_llm/common/tllmException.cpp`
- `sgl-kernel/3rdparty/tensorrt_llm/common/workspace.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/arch/copy_red_global.hpp`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/arch/mma.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/compute_occupancy.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue/collective/epilogue_moe_finalize.hpp`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue/thread/fused_activations.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue/threadblock/epilogue_per_row_per_col_scale.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue/threadblock/epilogue_tensor_op_int32.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/epilogue_helpers.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/builders/sm90_gmma_builder_gated.inl`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_builder_gated.hpp`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/collective_mma_gated.hpp`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_gated_tma_gmma_ss_warpspecialized.hpp`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/collective/sm90_mma_gated_tma_gmma_ss_warpspecialized_fp8.hpp`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/device/gemm_universal_base_compat.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/device/splitk_gemm_grouped.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/default_fpA_intB_traits.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/default_int8_traits.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/default_splitk_gemm_grouped.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/fpA_intB_gemm.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/fused_moe_kernel.cuh`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/fused_moe_kernel_routine.cuh`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/fused_moe_kernel_traits.cuh`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/gemm_moe_problem_visitor.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/gemm_universal_gated.hpp`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/gemm_with_epilogue_visitor.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/mixed_gemm_B_layout.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/moe_cute_util.cuh`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/moe_cutlass_kernel.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/moe_problem_visitor.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/sm90_gemm_gated_tma_warpspecialized_cooperative.hpp`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/sm90_gemm_gated_tma_warpspecialized_pingpong.hpp`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/kernel/splitk_gemm_grouped.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/default_dq_mma.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/default_dq_mma_multistage.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/default_dq_mma_pipelined.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/default_mma.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/default_mma_bf16.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_base.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_multistage.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_multistage_finegrained.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_multistage_percol.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_pipelined.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_pipelined_finegrained.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/threadblock/dq_mma_pipelined_percol.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/warp/default_mma_tensor_op.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/warp/mma_tensorop_compute_B_with_f16.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm/warp/mma_tensorop_dequantizer.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm_configs.h`
- `sgl-kernel/3rdparty/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/interleaved_numeric_conversion.h`

## Evidence Bundle

`evidence/pull-bundles/sglang/gh-3216`
