---
id: pr-flashinfer-2327
repo: flashinfer-ai/flashinfer
pr: 2327
title: '[perf] Improve gemm_fp8_nt_groupwise (cutlass backend) by 10-40% for batch sizes <= 32'
author: aidando73
date: '2026-01-14'
url: https://github.com/flashinfer-ai/flashinfer/pull/2327
source_category: upstream-code
architectures:
- sm100
tags:
- fp8
- gemm
techniques: []
hardware_features:
- fp8
kernel_types:
- gemm
languages:
- cuda-cpp
captured_at: '2026-04-27'
status: merged
merge_sha: 2bf87713dade8c6094ca3ff5e246e1ff72e97644
inclusion_reason: kernel file changes
changed_paths:
- benchmarks/bench_groupwise_gemm_fp8_blackwell.py
- csrc/gemm_groupwise_sm100.cu
- csrc/gemm_groupwise_sm100_kernel_inst.jinja
- include/flashinfer/gemm/gemm_groupwise_sm100.cuh
- tests/gemm/test_groupwise_scaled_gemm_fp8.py
head_ref_oid: 9a9cb9d885daf97249d8b08e0779b6aab68e07c1
artifact_dir: evidence/pull-bundles/flashinfer/gh-2327
---
# [perf] Improve gemm_fp8_nt_groupwise (cutlass backend) by 10-40% for batch sizes <= 32

- Repository: `flashinfer-ai/flashinfer`
- PR: [2327](https://github.com/flashinfer-ai/flashinfer/pull/2327)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `benchmarks/bench_groupwise_gemm_fp8_blackwell.py`
- `csrc/gemm_groupwise_sm100.cu`
- `csrc/gemm_groupwise_sm100_kernel_inst.jinja`
- `include/flashinfer/gemm/gemm_groupwise_sm100.cuh`
- `tests/gemm/test_groupwise_scaled_gemm_fp8.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-2327`
