---
id: pr-flashinfer-2914
repo: flashinfer-ai/flashinfer
pr: 2914
title: 'feat: Add cuBLASLt backend for `mm_bf16` and enable multi-tactic autotuning for FP8/MXFP8 runners'
author: vadiklyutiy
date: '2026-05-01'
url: https://github.com/flashinfer-ai/flashinfer/pull/2914
source_category: upstream-code
architectures:
- sm100
tags:
- benchmark_test
- fp4
- fp8
- gemm_quant
- memory_primitives
- mxfp4
- scheduler_autotune
- sm100
- topk
techniques:
- autotuning
- fine-grained-quantization
- tile-scheduling
hardware_features:
- blackwell
- fp4
- fp8
- mxfp4
kernel_types:
- gemm
- memory
- scheduler
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 770609920b31b120f74453cf6baf93c4f2a48cc0
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- benchmarks/routines/gemm.py
- csrc/bmm_fp8.cu
- csrc/flashinfer_gemm_binding.cu
- csrc/mm_bf16_cublaslt.cu
- flashinfer/aot.py
- flashinfer/autotuner.py
- flashinfer/gemm/gemm_base.py
- flashinfer/jit/gemm/__init__.py
- flashinfer/jit/gemm/core.py
- include/flashinfer/gemm/bmm_fp8.cuh
- include/flashinfer/gemm/mm_bf16_cublaslt.cuh
- tests/autotuner/test_autotuner_bmm_fp8.py
- tests/gemm/test_bmm_bf16.py
- tests/gemm/test_mm_bf16.py
artifact_dir: evidence/pull-bundles/flashinfer/gh-2914
head_ref_oid: 3e49178e50791b077cb62ae0e3e6f78b4813c7d4
---
# feat: Add cuBLASLt backend for `mm_bf16` and enable multi-tactic autotuning for FP8/MXFP8 runners

- Repository: `flashinfer-ai/flashinfer`
- PR: [2914](https://github.com/flashinfer-ai/flashinfer/pull/2914)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `benchmarks/routines/gemm.py`
- `csrc/bmm_fp8.cu`
- `csrc/flashinfer_gemm_binding.cu`
- `csrc/mm_bf16_cublaslt.cu`
- `flashinfer/aot.py`
- `flashinfer/autotuner.py`
- `flashinfer/gemm/gemm_base.py`
- `flashinfer/jit/gemm/__init__.py`
- `flashinfer/jit/gemm/core.py`
- `include/flashinfer/gemm/bmm_fp8.cuh`
- `include/flashinfer/gemm/mm_bf16_cublaslt.cuh`
- `tests/autotuner/test_autotuner_bmm_fp8.py`
- `tests/gemm/test_bmm_bf16.py`
- `tests/gemm/test_mm_bf16.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-2914`
