---
id: pr-flashinfer-748
repo: flashinfer-ai/flashinfer
pr: 748
title: '[Refactor] Unify JIT/Customization/AOT mode'
author: yzh119
date: '2025-01-23'
url: https://github.com/flashinfer-ai/flashinfer/pull/748
source_category: upstream-code
architectures:
- sm100
tags: []
techniques: []
hardware_features: []
kernel_types: []
languages: []
captured_at: '2026-05-16'
status: merged
merge_sha: 93e1a2634e22355b0856246b032b285ad1d1da6b
inclusion_reason: defer
changed_paths:
- CMakeLists.txt
- aot_build_utils/generate.py
- aot_build_utils/generate_aot_default_additional_params_header.py
- aot_build_utils/generate_batch_paged_decode_inst.py
- aot_build_utils/generate_batch_paged_prefill_inst.py
- aot_build_utils/generate_batch_paged_prefill_sm90_inst.py
- aot_build_utils/generate_batch_ragged_prefill_inst.py
- aot_build_utils/generate_batch_ragged_prefill_sm90_inst.py
- aot_build_utils/generate_dispatch_inc.py
- aot_build_utils/generate_single_decode_inst.py
- aot_build_utils/generate_single_prefill_inst.py
- aot_build_utils/generate_single_prefill_sm90_inst.py
- aot_build_utils/generate_sm90.py
- cmake/config.cmake
- csrc/aot_default_additional_params.h
- csrc/aot_extension_utils.h
- csrc/batch_decode.cu
- csrc/batch_decode_config.inc
- csrc/batch_decode_customize_config.jinja
- csrc/batch_decode_jit_pybind.cu
- csrc/batch_decode_kernel_inst.jinja
- csrc/batch_decode_mla_config.jinja
- csrc/batch_decode_mla_plan.cu
- csrc/batch_decode_mla_pybind.cu
- csrc/batch_decode_mla_run.cu
- csrc/batch_prefill.cu
- csrc/batch_prefill_config.inc
- csrc/batch_prefill_customize_config.jinja
- csrc/batch_prefill_jit_pybind.cu
- csrc/batch_prefill_paged_kernel_inst.jinja
- csrc/batch_prefill_paged_sm90_kernel_inst.jinja
- csrc/batch_prefill_ragged_kernel_inst.jinja
- csrc/batch_prefill_ragged_sm90_kernel_inst.jinja
- csrc/batch_prefill_sm90.cu
- csrc/batch_prefill_sm90_config.inc
- csrc/batch_prefill_sm90_customize_config.jinja
- csrc/batch_prefill_sm90_jit_pybind.cu
- csrc/flashinfer_ops.cu
- csrc/flashinfer_ops_sm90.cu
- csrc/single_decode.cu
- csrc/single_decode_config.inc
- csrc/single_decode_customize_config.jinja
- csrc/single_decode_jit_pybind.cu
- csrc/single_decode_kernel_inst.jinja
- csrc/single_prefill.cu
- csrc/single_prefill_config.inc
- csrc/single_prefill_customize_config.jinja
- csrc/single_prefill_jit_pybind.cu
- csrc/single_prefill_kernel_inst.jinja
- csrc/single_prefill_sm90.cu
- csrc/single_prefill_sm90_config.inc
- csrc/single_prefill_sm90_customize_config.jinja
- csrc/single_prefill_sm90_jit_pybind.cu
- csrc/single_prefill_sm90_kernel_inst.jinja
- docs/tutorials/kv_layout.rst
- flashinfer/cascade.py
- flashinfer/decode.py
- flashinfer/jit/__init__.py
- flashinfer/jit/attention.py
- flashinfer/jit/batch_decode_mla_templ.py
- flashinfer/jit/batch_decode_templ.py
- flashinfer/jit/batch_prefill_sm90_templ.py
- flashinfer/jit/batch_prefill_templ.py
- flashinfer/jit/core.py
- flashinfer/jit/single_decode_templ.py
- flashinfer/jit/single_prefill_sm90_templ.py
- flashinfer/jit/single_prefill_templ.py
- flashinfer/prefill.py
- flashinfer/sparse.py
- flashinfer/utils.py
- include/flashinfer/attention/decode.cuh
- include/flashinfer/attention/default_decode_params.cuh
- include/flashinfer/attention/default_prefill_params.cuh
- include/flashinfer/attention/hopper/attention_updater.cuh
- include/flashinfer/attention/hopper/default_params.cuh
- include/flashinfer/attention/hopper/kernel_traits.cuh
- include/flashinfer/attention/hopper/mainloop.cuh
- include/flashinfer/attention/hopper/prefill_sm90.cuh
- include/flashinfer/attention/hopper/sparse_mainloop.cuh
- include/flashinfer/attention/hopper/variants.cuh
- include/flashinfer/attention/prefill.cuh
- include/flashinfer/attention/scheduler.cuh
- include/flashinfer/attention/variants.cuh
- include/flashinfer/attention_impl.cuh
- include/flashinfer/fastdiv.cuh
- include/flashinfer/page.cuh
- include/flashinfer/utils.cuh
- setup.py
- src/bench_batch_decode.cu
- src/bench_batch_decode_mla.cu
- src/bench_batch_prefill.cu
- src/bench_cascade.cu
- src/bench_single_decode.cu
- src/bench_single_prefill.cu
- src/flashinfer_ops.cuh
- src/test_batch_decode.cu
- src/test_batch_prefill.cu
- src/test_cascade.cu
- src/test_single_prefill.cu
- src/tvm_wrapper.cu
- src/utils.h
- tests/jit_utils.py
- tests/test_alibi.py
- tests/test_batch_decode_kernels.py
- tests/test_batch_prefill_kernels.py
- tests/test_block_sparse.py
- tests/test_hopper.py
- tests/test_jit_example.py
- tests/test_jit_warmup.py
- tests/test_logits_cap.py
- tests/test_non_contiguous_decode.py
- tests/test_non_contiguous_prefill.py
- tests/test_shared_prefix_kernels.py
- tests/test_sliding_window.py
- tests/test_tensor_cores_decode.py
artifact_dir: evidence/pull-bundles/flashinfer/gh-748
head_ref_oid: e1d9238ffb86cdee58f31ff61712ef9d083c6a44
---
# [Refactor] Unify JIT/Customization/AOT mode

- Repository: `flashinfer-ai/flashinfer`
- PR: [748](https://github.com/flashinfer-ai/flashinfer/pull/748)
- Decision: `defer`
- Reason: defer

## Changed Paths

- `CMakeLists.txt`
- `aot_build_utils/generate.py`
- `aot_build_utils/generate_aot_default_additional_params_header.py`
- `aot_build_utils/generate_batch_paged_decode_inst.py`
- `aot_build_utils/generate_batch_paged_prefill_inst.py`
- `aot_build_utils/generate_batch_paged_prefill_sm90_inst.py`
- `aot_build_utils/generate_batch_ragged_prefill_inst.py`
- `aot_build_utils/generate_batch_ragged_prefill_sm90_inst.py`
- `aot_build_utils/generate_dispatch_inc.py`
- `aot_build_utils/generate_single_decode_inst.py`
- `aot_build_utils/generate_single_prefill_inst.py`
- `aot_build_utils/generate_single_prefill_sm90_inst.py`
- `aot_build_utils/generate_sm90.py`
- `cmake/config.cmake`
- `csrc/aot_default_additional_params.h`
- `csrc/aot_extension_utils.h`
- `csrc/batch_decode.cu`
- `csrc/batch_decode_config.inc`
- `csrc/batch_decode_customize_config.jinja`
- `csrc/batch_decode_jit_pybind.cu`
- `csrc/batch_decode_kernel_inst.jinja`
- `csrc/batch_decode_mla_config.jinja`
- `csrc/batch_decode_mla_plan.cu`
- `csrc/batch_decode_mla_pybind.cu`
- `csrc/batch_decode_mla_run.cu`
- `csrc/batch_prefill.cu`
- `csrc/batch_prefill_config.inc`
- `csrc/batch_prefill_customize_config.jinja`
- `csrc/batch_prefill_jit_pybind.cu`
- `csrc/batch_prefill_paged_kernel_inst.jinja`
- `csrc/batch_prefill_paged_sm90_kernel_inst.jinja`
- `csrc/batch_prefill_ragged_kernel_inst.jinja`
- `csrc/batch_prefill_ragged_sm90_kernel_inst.jinja`
- `csrc/batch_prefill_sm90.cu`
- `csrc/batch_prefill_sm90_config.inc`
- `csrc/batch_prefill_sm90_customize_config.jinja`
- `csrc/batch_prefill_sm90_jit_pybind.cu`
- `csrc/flashinfer_ops.cu`
- `csrc/flashinfer_ops_sm90.cu`
- `csrc/single_decode.cu`
- `csrc/single_decode_config.inc`
- `csrc/single_decode_customize_config.jinja`
- `csrc/single_decode_jit_pybind.cu`
- `csrc/single_decode_kernel_inst.jinja`
- `csrc/single_prefill.cu`
- `csrc/single_prefill_config.inc`
- `csrc/single_prefill_customize_config.jinja`
- `csrc/single_prefill_jit_pybind.cu`
- `csrc/single_prefill_kernel_inst.jinja`
- `csrc/single_prefill_sm90.cu`
- `csrc/single_prefill_sm90_config.inc`
- `csrc/single_prefill_sm90_customize_config.jinja`
- `csrc/single_prefill_sm90_jit_pybind.cu`
- `csrc/single_prefill_sm90_kernel_inst.jinja`
- `docs/tutorials/kv_layout.rst`
- `flashinfer/cascade.py`
- `flashinfer/decode.py`
- `flashinfer/jit/__init__.py`
- `flashinfer/jit/attention.py`
- `flashinfer/jit/batch_decode_mla_templ.py`
- `flashinfer/jit/batch_decode_templ.py`
- `flashinfer/jit/batch_prefill_sm90_templ.py`
- `flashinfer/jit/batch_prefill_templ.py`
- `flashinfer/jit/core.py`
- `flashinfer/jit/single_decode_templ.py`
- `flashinfer/jit/single_prefill_sm90_templ.py`
- `flashinfer/jit/single_prefill_templ.py`
- `flashinfer/prefill.py`
- `flashinfer/sparse.py`
- `flashinfer/utils.py`
- `include/flashinfer/attention/decode.cuh`
- `include/flashinfer/attention/default_decode_params.cuh`
- `include/flashinfer/attention/default_prefill_params.cuh`
- `include/flashinfer/attention/hopper/attention_updater.cuh`
- `include/flashinfer/attention/hopper/default_params.cuh`
- `include/flashinfer/attention/hopper/kernel_traits.cuh`
- `include/flashinfer/attention/hopper/mainloop.cuh`
- `include/flashinfer/attention/hopper/prefill_sm90.cuh`
- `include/flashinfer/attention/hopper/sparse_mainloop.cuh`
- `include/flashinfer/attention/hopper/variants.cuh`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-748`
