---
id: pr-flashinfer-3059
repo: flashinfer-ai/flashinfer
pr: 3059
title: Support Allreduce + Norm + Per-token Group Fp8 Quant Fusion
author: wzhao18
date: '2026-05-06'
url: https://github.com/flashinfer-ai/flashinfer/pull/3059
source_category: upstream-code
architectures:
- sm100
tags:
- TMA
- fp8
- gemm_quant
- memory_primitives
- moe_routing
- norm_elementwise
techniques:
- async-copy
- fine-grained-quantization
- kernel-fusion
hardware_features:
- fp8
- tma
kernel_types:
- gemm
- memory
- moe
- norm
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 435826ab05a3332209d138004a693dbc1003d6b8
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- csrc/trtllm_allreduce_fusion.cu
- flashinfer/comm/allreduce.py
- flashinfer/comm/trtllm_ar.py
- include/flashinfer/comm/trtllm_allreduce_fusion.cuh
- tests/comm/test_trtllm_allreduce_fusion_group_fp8_quant.py
artifact_dir: evidence/pull-bundles/flashinfer/gh-3059
head_ref_oid: e8520f459d718a0b9ccc444d0fd292d5de2271f9
---
# Support Allreduce + Norm + Per-token Group Fp8 Quant Fusion

- Repository: `flashinfer-ai/flashinfer`
- PR: [3059](https://github.com/flashinfer-ai/flashinfer/pull/3059)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `csrc/trtllm_allreduce_fusion.cu`
- `flashinfer/comm/allreduce.py`
- `flashinfer/comm/trtllm_ar.py`
- `include/flashinfer/comm/trtllm_allreduce_fusion.cuh`
- `tests/comm/test_trtllm_allreduce_fusion_group_fp8_quant.py`

## Evidence Bundle

`evidence/pull-bundles/flashinfer/gh-3059`
