---
id: pr-flash-attention-1100
repo: Dao-AILab/flash-attention
pr: 1100
title: Fp8 kernel with "in-kernel" transpose of V in producer
author: jayhshah
date: '2024-07-30'
url: https://github.com/Dao-AILab/flash-attention/pull/1100
source_category: upstream-code
architectures:
- sm90
tags:
- arch_pipeline
- attention_kv
- benchmark_test
- fp8
- gemm_quant
- memory_primitives
techniques:
- async-copy
- epilogue-fusion
- fine-grained-quantization
- pipeline-stages
- tile-scheduling
- warp-specialization
hardware_features:
- fp8
- hopper
- tma
kernel_types:
- attention
- gemm
- memory
- runtime
- softmax
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 5018ac6ac531aabdb05c8af1ba3d98a2235bdbde
inclusion_reason: Mine shape sets, tolerance rules, warmup logic, and profile commands before mutating code.
changed_paths:
- hopper/benchmark_flash_attention_fp8.py
- hopper/epilogue_fwd_sm90_tma.hpp
- hopper/flash_api.cpp
- hopper/flash_fwd_hdim128_e4m3_sm90.cu
- hopper/flash_fwd_hdim256_e4m3_sm90.cu
- hopper/flash_fwd_hdim64_e4m3_sm90.cu
- hopper/flash_fwd_kernel.h
- hopper/flash_fwd_launch_template.h
- hopper/kernel_traits.h
- hopper/mainloop_fwd_sm90_tma_gmma_ws.hpp
- hopper/named_barrier.hpp
- hopper/setup.py
- hopper/softmax.h
- hopper/test_flash_attn.py
- hopper/tile_scheduler.hpp
- hopper/utils.h
artifact_dir: evidence/pull-bundles/flash-attention/gh-1100
head_ref_oid: 6b468f8406dff9026c4d834abc5a8eb6d01f6584
---
# Fp8 kernel with "in-kernel" transpose of V in producer

- Repository: `Dao-AILab/flash-attention`
- PR: [1100](https://github.com/Dao-AILab/flash-attention/pull/1100)
- Decision: `Mine shape sets, tolerance rules, warmup logic, and profile commands before mutating code.`
- Reason: Mine shape sets, tolerance rules, warmup logic, and profile commands before mutating code.

## Changed Paths

- `hopper/benchmark_flash_attention_fp8.py`
- `hopper/epilogue_fwd_sm90_tma.hpp`
- `hopper/flash_api.cpp`
- `hopper/flash_fwd_hdim128_e4m3_sm90.cu`
- `hopper/flash_fwd_hdim256_e4m3_sm90.cu`
- `hopper/flash_fwd_hdim64_e4m3_sm90.cu`
- `hopper/flash_fwd_kernel.h`
- `hopper/flash_fwd_launch_template.h`
- `hopper/kernel_traits.h`
- `hopper/mainloop_fwd_sm90_tma_gmma_ws.hpp`
- `hopper/named_barrier.hpp`
- `hopper/setup.py`
- `hopper/softmax.h`
- `hopper/test_flash_attn.py`
- `hopper/tile_scheduler.hpp`
- `hopper/utils.h`

## Evidence Bundle

`evidence/pull-bundles/flash-attention/gh-1100`
