---
id: pr-flashinfer-969
repo: flashinfer-ai/flashinfer
pr: 969
title: 'perf: Fix python API overhead when CUDAGraph is not enabled'
author: yzh119
date: '2025-03-24'
url: https://github.com/flashinfer-ai/flashinfer/pull/969
source_category: upstream-code
architectures:
- sm100
tags:
- decode
- fp8
- gemm
- mla
- prefill
- quantization
techniques: []
hardware_features:
- fp8
kernel_types:
- decode
- gemm
- mla
- prefill
- quantization
languages:
- cuda-cpp
- python
captured_at: '2026-04-17'
status: merged
merge_sha: 61e049a02e18c67f0dfb8b65ffb67060650107df
inclusion_reason: kernel file changes
changed_paths:
- csrc/activation.cu
- csrc/batch_decode.cu
- csrc/batch_decode_jit_pybind.cu
- csrc/batch_mla_plan.cu
- csrc/batch_mla_pybind.cu
- csrc/batch_mla_run.cu
- csrc/batch_mla_sm90_plan.cu
- csrc/batch_mla_sm90_pybind.cu
- csrc/batch_mla_sm90_run.cu
- csrc/batch_prefill.cu
- csrc/batch_prefill_jit_pybind.cu
- csrc/batch_prefill_sm90.cu
- csrc/batch_prefill_sm90_jit_pybind.cu
- csrc/bmm_fp8.cu
- csrc/cascade.cu
- csrc/flashinfer_cascade_ops.cu
- csrc/flashinfer_gemm_ops.cu
- csrc/flashinfer_gemm_sm90_ops.cu
- csrc/flashinfer_norm_ops.cu
- csrc/flashinfer_ops.cu
- csrc/flashinfer_ops_sm90.cu
- csrc/flashinfer_page_ops.cu
- csrc/flashinfer_quantization_ops.cu
- csrc/flashinfer_rope_ops.cu
- csrc/flashinfer_sampling_ops.cu
- csrc/group_gemm.cu
- csrc/group_gemm_sm90.cu
- csrc/norm.cu
- csrc/page.cu
- csrc/pod.cu
- csrc/pod_jit_pybind.cu
- csrc/pytorch_extension_utils.h
- csrc/quantization.cu
- csrc/renorm.cu
- csrc/rope.cu
- csrc/sampling.cu
- csrc/single_decode.cu
- csrc/single_decode_jit_pybind.cu
- csrc/single_prefill.cu
- csrc/single_prefill_jit_pybind.cu
- csrc/single_prefill_sm90.cu
- csrc/single_prefill_sm90_jit_pybind.cu
- flashinfer/activation.py
- flashinfer/cascade.py
- flashinfer/decode.py
- flashinfer/gemm.py
- flashinfer/jit/activation.py
- flashinfer/mla.py
- flashinfer/norm.py
- flashinfer/page.py
- flashinfer/pod.py
- flashinfer/prefill.py
- flashinfer/quantization.py
- flashinfer/rope.py
- flashinfer/sampling.py
- flashinfer/sparse.py
- flashinfer/utils.py
- scripts/task_jit_run_tests_part4.sh
- tests/test_group_gemm.py
head_ref_oid: 82484b7f34d10c5f746c3b943dffdcf4a34414a0
artifact_dir: evidence/pull-bundles/flashinfer/gh-969
---
# perf: Fix python API overhead when CUDAGraph is not enabled

- Repository: `flashinfer-ai/flashinfer`
- PR: [969](https://github.com/flashinfer-ai/flashinfer/pull/969)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `csrc/activation.cu`
- `csrc/batch_decode.cu`
- `csrc/batch_decode_jit_pybind.cu`
- `csrc/batch_mla_plan.cu`
- `csrc/batch_mla_pybind.cu`
- `csrc/batch_mla_run.cu`
- `csrc/batch_mla_sm90_plan.cu`
- `csrc/batch_mla_sm90_pybind.cu`
- `csrc/batch_mla_sm90_run.cu`
- `csrc/batch_prefill.cu`
- `csrc/batch_prefill_jit_pybind.cu`
- `csrc/batch_prefill_sm90.cu`
- `csrc/batch_prefill_sm90_jit_pybind.cu`
- `csrc/bmm_fp8.cu`
- `csrc/cascade.cu`
- `csrc/flashinfer_cascade_ops.cu`
- `csrc/flashinfer_gemm_ops.cu`
- `csrc/flashinfer_gemm_sm90_ops.cu`
- `csrc/flashinfer_norm_ops.cu`
- `csrc/flashinfer_ops.cu`
- `csrc/flashinfer_ops_sm90.cu`
- `csrc/flashinfer_page_ops.cu`
- `csrc/flashinfer_quantization_ops.cu`
- `csrc/flashinfer_rope_ops.cu`
- `csrc/flashinfer_sampling_ops.cu`
- `csrc/group_gemm.cu`
- `csrc/group_gemm_sm90.cu`
- `csrc/norm.cu`
- `csrc/page.cu`
- `csrc/pod.cu`
- `csrc/pod_jit_pybind.cu`
- `csrc/pytorch_extension_utils.h`
- `csrc/quantization.cu`
- `csrc/renorm.cu`
- `csrc/rope.cu`
- `csrc/sampling.cu`
- `csrc/single_decode.cu`
- `csrc/single_decode_jit_pybind.cu`
- `csrc/single_prefill.cu`
- `csrc/single_prefill_jit_pybind.cu`
- `csrc/single_prefill_sm90.cu`
- `csrc/single_prefill_sm90_jit_pybind.cu`
- `flashinfer/activation.py`
- `flashinfer/cascade.py`
- `flashinfer/decode.py`
- `flashinfer/gemm.py`
- `flashinfer/jit/activation.py`
- `flashinfer/mla.py`
- `flashinfer/norm.py`
- `flashinfer/page.py`
- `flashinfer/pod.py`
- `flashinfer/prefill.py`
- `flashinfer/quantization.py`
- `flashinfer/rope.py`
- `flashinfer/sampling.py`
- `flashinfer/sparse.py`
- `flashinfer/utils.py`
- `scripts/task_jit_run_tests_part4.sh`
- `tests/test_group_gemm.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-969`
