---
id: pr-flashinfer-1608
repo: flashinfer-ai/flashinfer
pr: 1608
title: 'feat: initial support for SM103, SM110, SM120, SM121'
author: aleozlx
date: '2025-09-02'
url: https://github.com/flashinfer-ai/flashinfer/pull/1608
source_category: upstream-code
architectures:
- sm100
tags:
- attention
- flash-attention
- fp4
- fp8
- gemm
- kernel-fusion
- mla
- moe
- prefill
- quantization
- tma
techniques:
- kernel-fusion
hardware_features:
- fp4
- fp8
- tma
kernel_types:
- attention
- flash-attention
- gemm
- mla
- moe
- prefill
- quantization
languages:
- cuda-cpp
- cute-dsl
captured_at: '2026-04-27'
status: merged
merge_sha: beebc6074c37353445a27106eef3f7077d718ec4
inclusion_reason: kernel file changes
changed_paths:
- .github/workflows/release_wheel_aarch64.yml
- .github/workflows/release_wheel_sglang_x86_64.yml
- .github/workflows/release_wheel_x86_64.yml
- README.md
- csrc/group_gemm_mxfp4_groupwise_sm100.cu
- csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm_configs.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_template_dispatch.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_template_dispatch_tma_ws.h
- csrc/pytorch_extension_utils.h
- csrc/trtllm_fused_moe_kernel_launcher.cu
- csrc/trtllm_gemm_runner.cu
- docs/installation.rst
- flashinfer/aot.py
- flashinfer/artifacts.py
- flashinfer/comm/trtllm_ar.py
- flashinfer/compilation_context.py
- flashinfer/cute_dsl/blockscaled_gemm.py
- flashinfer/fp4_quantization.py
- flashinfer/fp8_quantization.py
- flashinfer/fused_moe/core.py
- flashinfer/gemm.py
- flashinfer/jit/__init__.py
- flashinfer/jit/attention/pytorch.py
- flashinfer/jit/core.py
- flashinfer/jit/cpp_ext.py
- flashinfer/jit/env.py
- flashinfer/mla.py
- flashinfer/prefill.py
- flashinfer/utils.py
- include/flashinfer/gemm/cutlass_gemm_configs.h
- include/flashinfer/gemm/fp4_gemm_template_sm100.h
- include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/BatchedGemmInterface.h
- include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/BatchedGemmOptions.h
- include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/GemmGatedActOptions.h
- include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/GemmOptions.h
- include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/KernelTraits.h
- include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/trtllm/gen/MmaDecl.h
- include/flashinfer/trtllm/common.h
- include/flashinfer/trtllm/fmha/fmhaKernels.cuh
- include/flashinfer/trtllm/fmha/fmhaRunner.cuh
- include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/GemmInterface.h
- include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/GemmOptions.h
- include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/KernelParams.h
- include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/KernelParamsDecl.h
- include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/KernelTraits.h
- include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/TmaDescriptor.h
- include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/trtllm/gen/MmaDecl.h
- include/flashinfer/vec_dtypes.cuh
- scripts/run-ci-build-wheel.sh
- scripts/task_test_aot_build_import.sh
- setup.py
- tests/test_batch_attention.py
- tests/test_blackwell_fmha.py
- tests/test_cute_dsl_blockscaled_gemm.py
- tests/test_deepseek_mla.py
- tests/test_groupwise_scaled_gemm_mxfp4.py
- tests/test_mm_fp4.py
- tests/test_trtllm_cutlass_fused_moe.py
head_ref_oid: 7e2d57ec87c80f98fd24f9c72eb7ecff59b1710c
artifact_dir: evidence/pull-bundles/flashinfer/gh-1608
---
# feat: initial support for SM103, SM110, SM120, SM121

- Repository: `flashinfer-ai/flashinfer`
- PR: [1608](https://github.com/flashinfer-ai/flashinfer/pull/1608)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `.github/workflows/release_wheel_aarch64.yml`
- `.github/workflows/release_wheel_sglang_x86_64.yml`
- `.github/workflows/release_wheel_x86_64.yml`
- `README.md`
- `csrc/group_gemm_mxfp4_groupwise_sm100.cu`
- `csrc/nv_internal/tensorrt_llm/cutlass_extensions/include/cutlass_extensions/gemm_configs.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/cutlass_heuristic.cpp`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_template_dispatch.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_gemm_template_dispatch_tma_ws.h`
- `csrc/pytorch_extension_utils.h`
- `csrc/trtllm_fused_moe_kernel_launcher.cu`
- `csrc/trtllm_gemm_runner.cu`
- `docs/installation.rst`
- `flashinfer/aot.py`
- `flashinfer/artifacts.py`
- `flashinfer/comm/trtllm_ar.py`
- `flashinfer/compilation_context.py`
- `flashinfer/cute_dsl/blockscaled_gemm.py`
- `flashinfer/fp4_quantization.py`
- `flashinfer/fp8_quantization.py`
- `flashinfer/fused_moe/core.py`
- `flashinfer/gemm.py`
- `flashinfer/jit/__init__.py`
- `flashinfer/jit/attention/pytorch.py`
- `flashinfer/jit/core.py`
- `flashinfer/jit/cpp_ext.py`
- `flashinfer/jit/env.py`
- `flashinfer/mla.py`
- `flashinfer/prefill.py`
- `flashinfer/utils.py`
- `include/flashinfer/gemm/cutlass_gemm_configs.h`
- `include/flashinfer/gemm/fp4_gemm_template_sm100.h`
- `include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/BatchedGemmInterface.h`
- `include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/BatchedGemmOptions.h`
- `include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/GemmGatedActOptions.h`
- `include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/GemmOptions.h`
- `include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/KernelTraits.h`
- `include/flashinfer/trtllm/batched_gemm/trtllmGen_bmm_export/trtllm/gen/MmaDecl.h`
- `include/flashinfer/trtllm/common.h`
- `include/flashinfer/trtllm/fmha/fmhaKernels.cuh`
- `include/flashinfer/trtllm/fmha/fmhaRunner.cuh`
- `include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/GemmInterface.h`
- `include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/GemmOptions.h`
- `include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/KernelParams.h`
- `include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/KernelParamsDecl.h`
- `include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/KernelTraits.h`
- `include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/TmaDescriptor.h`
- `include/flashinfer/trtllm/gemm/trtllmGen_gemm_export/trtllm/gen/MmaDecl.h`
- `include/flashinfer/vec_dtypes.cuh`
- `scripts/run-ci-build-wheel.sh`
- `scripts/task_test_aot_build_import.sh`
- `setup.py`
- `tests/test_batch_attention.py`
- `tests/test_blackwell_fmha.py`
- `tests/test_cute_dsl_blockscaled_gemm.py`
- `tests/test_deepseek_mla.py`
- `tests/test_groupwise_scaled_gemm_mxfp4.py`
- `tests/test_mm_fp4.py`
- `tests/test_trtllm_cutlass_fused_moe.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-1608`
