---
id: pr-vllm-41326
repo: vllm-project/vllm
pr: 41326
title: Faster per-token fp8 group quant packed kernel for blackwell
author: zyongye
date: '2026-05-01'
url: https://github.com/vllm-project/vllm/pull/41326
source_category: upstream-code
architectures:
- sm100
tags:
- CUDA kernel
- DeepGEMM
- benchmark kernel
- blackwell
- csrc kernel
- fp8
- gemm_quant
- quantization kernel
techniques:
- fine-grained-quantization
hardware_features:
- blackwell
- fp8
kernel_types:
- gemm
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: a3c83ff2fd050bc6392260d6e84ee1150f238f26
inclusion_reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.
changed_paths:
- csrc/libtorch_stable/quantization/w8a8/fp8/per_token_group_quant.cu
- csrc/libtorch_stable/quantization/w8a8/per_token_group_quant_8bit.h
- tests/kernels/quantization/test_per_token_group_quant.py
artifact_dir: evidence/pull-bundles/vllm/gh-41326
head_ref_oid: 8009fe01fe87267446ed7128f3a0a6b880652a03
---
# Faster per-token fp8 group quant packed kernel for blackwell

- Repository: `vllm-project/vllm`
- PR: [41326](https://github.com/vllm-project/vllm/pull/41326)
- Decision: `Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.`
- Reason: Inspect scale layout, accumulator type, tile/schedule choice, epilogue fusion, and partial-tile guards before deriving a candidate.

## Changed Paths

- `csrc/libtorch_stable/quantization/w8a8/fp8/per_token_group_quant.cu`
- `csrc/libtorch_stable/quantization/w8a8/per_token_group_quant_8bit.h`
- `tests/kernels/quantization/test_per_token_group_quant.py`

## Evidence Bundle

`evidence/pull-bundles/vllm/gh-41326`
