---
id: pr-flashinfer-1641
repo: flashinfer-ai/flashinfer
pr: 1641
title: 'refactor: using tvm-ffi for multi-platform bindings'
author: yzh119
date: '2025-09-27'
url: https://github.com/flashinfer-ai/flashinfer/pull/1641
source_category: upstream-code
architectures:
- sm100
tags: []
techniques: []
hardware_features: []
kernel_types: []
languages: []
captured_at: '2026-05-16'
status: merged
merge_sha: 86d3e1360d7ac0541adf696c2dc9f9ff308e8c4e
inclusion_reason: defer
changed_paths:
- ci/bash.sh
- csrc/activation.cu
- csrc/aot_extension_utils.h
- csrc/batch_attention.cu
- csrc/batch_attention_jit_pybind.cu
- csrc/batch_decode.cu
- csrc/batch_decode_config.inc
- csrc/batch_decode_jit_pybind.cu
- csrc/batch_decode_mla_cute_sm80.cu
- csrc/batch_decode_mla_plan.cu
- csrc/batch_decode_mla_pybind.cu
- csrc/batch_decode_mla_run.cu
- csrc/batch_mla_config.jinja
- csrc/batch_mla_plan.cu
- csrc/batch_mla_pybind.cu
- csrc/batch_mla_run.cu
- csrc/batch_mla_sm90_plan.cu
- csrc/batch_mla_sm90_pybind.cu
- csrc/batch_mla_sm90_run.cu
- csrc/batch_prefill.cu
- csrc/batch_prefill_config.inc
- csrc/batch_prefill_fp8_sm90.cu
- csrc/batch_prefill_jit_pybind.cu
- csrc/batch_prefill_sm90.cu
- csrc/batch_prefill_sm90_config.inc
- csrc/batch_prefill_sm90_customize_config.jinja
- csrc/batch_prefill_sm90_jit_pybind.cu
- csrc/blackwell_fmha_plan.cu
- csrc/bmm_fp8.cu
- csrc/cascade.cu
- csrc/cudnn_sdpa_kernel_launcher.cu
- csrc/cutlass_mla.cu
- csrc/flashinfer_cascade_ops.cu
- csrc/flashinfer_gemm_ops.cu
- csrc/flashinfer_gemm_sm90_ops.cu
- csrc/flashinfer_mla_ops.cu
- csrc/flashinfer_norm_ops.cu
- csrc/flashinfer_page_binding.cu
- csrc/flashinfer_page_ops.cu
- csrc/flashinfer_quantization_ops.cu
- csrc/flashinfer_rope_ops.cu
- csrc/flashinfer_sampling_ops.cu
- csrc/flashinfer_xqa_binding.cu
- csrc/fmha_cutlass_sm100.cu
- csrc/fmha_cutlass_sm100_pybind.cu
- csrc/fp4_gemm_cutlass.cu
- csrc/fp4_gemm_cutlass_sm120.cu
- csrc/fp8_gemm_cutlass.cu
- csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_ops.cu
- csrc/gemm_groupwise_sm100.cu
- csrc/gemm_groupwise_sm120.cu
- csrc/gemm_sm100_pybind.cu
- csrc/gemm_sm120_pybind.cu
- csrc/group_gemm.cu
- csrc/group_gemm_fp8_groupwise_sm100.cu
- csrc/group_gemm_fp8_groupwise_sm120.cu
- csrc/group_gemm_mxfp4_groupwise_sm100.cu
- csrc/group_gemm_sm100_pybind.cu
- csrc/group_gemm_sm120_pybind.cu
- csrc/group_gemm_sm90.cu
- csrc/logging.cc
- csrc/norm.cu
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/include/moe_gemm_kernels.h
- csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/include/moe_util_kernels.h
- csrc/nv_internal/tensorrt_llm/kernels/delayStream.cu
- csrc/nv_internal/tensorrt_llm/runtime/torchUtils.h
- csrc/nv_internal/tensorrt_llm/thop/fp4Op.cpp
- csrc/nv_internal/tensorrt_llm/thop/fp4Quantize.cpp
- csrc/nv_internal/tensorrt_llm/thop/fp4Quantize.h
- csrc/nv_internal/tensorrt_llm/thop/fp8Quantize.cpp
- csrc/nv_internal/tensorrt_llm/thop/fp8Quantize.h
- csrc/nv_internal/tensorrt_llm/thop/thUtils.h
- csrc/nv_internal/tensorrt_llm/thop/utils.h
- csrc/nvshmem_binding.cu
- csrc/page.cu
- csrc/pod.cu
- csrc/pod_config.inc
- csrc/pod_jit_pybind.cu
- csrc/pod_kernel_inst.jinja
- csrc/pytorch_extension_utils.h
- csrc/quantization.cu
- csrc/renorm.cu
- csrc/rope.cu
- csrc/sampling.cu
- csrc/single_decode.cu
- csrc/single_decode_config.inc
- csrc/single_decode_jit_pybind.cu
- csrc/single_prefill.cu
- csrc/single_prefill_config.inc
- csrc/single_prefill_fp8_sm90.cu
- csrc/single_prefill_jit_pybind.cu
- csrc/single_prefill_sm90.cu
- csrc/single_prefill_sm90_config.inc
- csrc/single_prefill_sm90_jit_pybind.cu
- csrc/tgv_gemm.cu
- csrc/trtllm_allreduce.cu
- csrc/trtllm_allreduce_fusion.cu
- csrc/trtllm_alltoall.cu
- csrc/trtllm_fmha_kernel_launcher.cu
- csrc/trtllm_fused_moe_kernel_launcher.cu
- csrc/trtllm_fused_moe_routing_renormalize.cu
- csrc/trtllm_gemm_runner.cu
- csrc/trtllm_mnnvl_allreduce.cu
- csrc/trtllm_moe_allreduce_fusion.cu
- csrc/tvm_ffi_utils.h
- csrc/vllm_custom_all_reduce.cu
- csrc/xqa/xqa_wrapper.cu
- custom_backend.py
- flashinfer/__main__.py
- flashinfer/activation.py
- flashinfer/comm/nvshmem.py
- flashinfer/comm/nvshmem_allreduce.py
- flashinfer/comm/trtllm_alltoall.py
- flashinfer/comm/trtllm_mnnvl_ar.py
- flashinfer/comm/vllm_ar.py
- flashinfer/decode.py
- flashinfer/fp4_quantization.py
- flashinfer/fp8_quantization.py
- flashinfer/fused_moe/core.py
- flashinfer/gemm.py
artifact_dir: evidence/pull-bundles/flashinfer/gh-1641
head_ref_oid: d76f4c3ce726180b46ac6bd324d5d0f68cca06f7
---
# refactor: using tvm-ffi for multi-platform bindings

- Repository: `flashinfer-ai/flashinfer`
- PR: [1641](https://github.com/flashinfer-ai/flashinfer/pull/1641)
- Decision: `defer`
- Reason: defer

## Changed Paths

- `ci/bash.sh`
- `csrc/activation.cu`
- `csrc/aot_extension_utils.h`
- `csrc/batch_attention.cu`
- `csrc/batch_attention_jit_pybind.cu`
- `csrc/batch_decode.cu`
- `csrc/batch_decode_config.inc`
- `csrc/batch_decode_jit_pybind.cu`
- `csrc/batch_decode_mla_cute_sm80.cu`
- `csrc/batch_decode_mla_plan.cu`
- `csrc/batch_decode_mla_pybind.cu`
- `csrc/batch_decode_mla_run.cu`
- `csrc/batch_mla_config.jinja`
- `csrc/batch_mla_plan.cu`
- `csrc/batch_mla_pybind.cu`
- `csrc/batch_mla_run.cu`
- `csrc/batch_mla_sm90_plan.cu`
- `csrc/batch_mla_sm90_pybind.cu`
- `csrc/batch_mla_sm90_run.cu`
- `csrc/batch_prefill.cu`
- `csrc/batch_prefill_config.inc`
- `csrc/batch_prefill_fp8_sm90.cu`
- `csrc/batch_prefill_jit_pybind.cu`
- `csrc/batch_prefill_sm90.cu`
- `csrc/batch_prefill_sm90_config.inc`
- `csrc/batch_prefill_sm90_customize_config.jinja`
- `csrc/batch_prefill_sm90_jit_pybind.cu`
- `csrc/blackwell_fmha_plan.cu`
- `csrc/bmm_fp8.cu`
- `csrc/cascade.cu`
- `csrc/cudnn_sdpa_kernel_launcher.cu`
- `csrc/cutlass_mla.cu`
- `csrc/flashinfer_cascade_ops.cu`
- `csrc/flashinfer_gemm_ops.cu`
- `csrc/flashinfer_gemm_sm90_ops.cu`
- `csrc/flashinfer_mla_ops.cu`
- `csrc/flashinfer_norm_ops.cu`
- `csrc/flashinfer_page_binding.cu`
- `csrc/flashinfer_page_ops.cu`
- `csrc/flashinfer_quantization_ops.cu`
- `csrc/flashinfer_rope_ops.cu`
- `csrc/flashinfer_sampling_ops.cu`
- `csrc/flashinfer_xqa_binding.cu`
- `csrc/fmha_cutlass_sm100.cu`
- `csrc/fmha_cutlass_sm100_pybind.cu`
- `csrc/fp4_gemm_cutlass.cu`
- `csrc/fp4_gemm_cutlass_sm120.cu`
- `csrc/fp8_gemm_cutlass.cu`
- `csrc/fused_moe/cutlass_backend/flashinfer_cutlass_fused_moe_sm100_ops.cu`
- `csrc/gemm_groupwise_sm100.cu`
- `csrc/gemm_groupwise_sm120.cu`
- `csrc/gemm_sm100_pybind.cu`
- `csrc/gemm_sm120_pybind.cu`
- `csrc/group_gemm.cu`
- `csrc/group_gemm_fp8_groupwise_sm100.cu`
- `csrc/group_gemm_fp8_groupwise_sm120.cu`
- `csrc/group_gemm_mxfp4_groupwise_sm100.cu`
- `csrc/group_gemm_sm100_pybind.cu`
- `csrc/group_gemm_sm120_pybind.cu`
- `csrc/group_gemm_sm90.cu`
- `csrc/logging.cc`
- `csrc/norm.cu`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/include/moe_gemm_kernels.h`
- `csrc/nv_internal/tensorrt_llm/kernels/cutlass_kernels/include/moe_util_kernels.h`
- `csrc/nv_internal/tensorrt_llm/kernels/delayStream.cu`
- `csrc/nv_internal/tensorrt_llm/runtime/torchUtils.h`
- `csrc/nv_internal/tensorrt_llm/thop/fp4Op.cpp`
- `csrc/nv_internal/tensorrt_llm/thop/fp4Quantize.cpp`
- `csrc/nv_internal/tensorrt_llm/thop/fp4Quantize.h`
- `csrc/nv_internal/tensorrt_llm/thop/fp8Quantize.cpp`
- `csrc/nv_internal/tensorrt_llm/thop/fp8Quantize.h`
- `csrc/nv_internal/tensorrt_llm/thop/thUtils.h`
- `csrc/nv_internal/tensorrt_llm/thop/utils.h`
- `csrc/nvshmem_binding.cu`
- `csrc/page.cu`
- `csrc/pod.cu`
- `csrc/pod_config.inc`
- `csrc/pod_jit_pybind.cu`
- `csrc/pod_kernel_inst.jinja`
- `csrc/pytorch_extension_utils.h`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-1641`
