---
id: pr-flash-attention-1823
repo: Dao-AILab/flash-attention
pr: 1823
title: Add sorting and head swizzle to varlen scheduler
author: jayhshah
date: '2025-08-22'
url: https://github.com/Dao-AILab/flash-attention/pull/1823
source_category: upstream-code
architectures:
- sm90
tags:
- FA3
- arch_pipeline
- attention_kv
- memory_primitives
- scheduler_autotune
techniques:
- autotuning
- pipeline-stages
- swizzling
- tile-scheduling
hardware_features:
- hopper
kernel_types:
- attention
- memory
- runtime
- scheduler
- sort
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 199401d31f940d1f062eb9c0233b41ef62baa5ae
inclusion_reason: Treat scheduler/autotune configs as shape-specific evidence; replay benchmark shapes before generalizing.
changed_paths:
- hopper/flash.h
- hopper/flash_api.cpp
- hopper/flash_attn_interface.py
- hopper/flash_fwd_combine_kernel.h
- hopper/flash_fwd_combine_launch_template.h
- hopper/flash_fwd_launch_template.h
- hopper/flash_prepare_scheduler.cu
- hopper/setup.py
- hopper/static_switch.h
- hopper/test_flash_attn.py
- hopper/tile_scheduler.hpp
- hopper/tile_size.h
artifact_dir: evidence/pull-bundles/flash-attention/gh-1823
head_ref_oid: 3024f683c465336baa231ef467972bfe45d5b672
---
# Add sorting and head swizzle to varlen scheduler

- Repository: `Dao-AILab/flash-attention`
- PR: [1823](https://github.com/Dao-AILab/flash-attention/pull/1823)
- Decision: `Treat scheduler/autotune configs as shape-specific evidence; replay benchmark shapes before generalizing.`
- Reason: Treat scheduler/autotune configs as shape-specific evidence; replay benchmark shapes before generalizing.

## Changed Paths

- `hopper/flash.h`
- `hopper/flash_api.cpp`
- `hopper/flash_attn_interface.py`
- `hopper/flash_fwd_combine_kernel.h`
- `hopper/flash_fwd_combine_launch_template.h`
- `hopper/flash_fwd_launch_template.h`
- `hopper/flash_prepare_scheduler.cu`
- `hopper/setup.py`
- `hopper/static_switch.h`
- `hopper/test_flash_attn.py`
- `hopper/tile_scheduler.hpp`
- `hopper/tile_size.h`

## Evidence Bundle

`evidence/pull-bundles/flash-attention/gh-1823`
