---
id: pr-flash-attention-1025
repo: Dao-AILab/flash-attention
pr: 1025
title: '[WIP] First draft for softcapping.'
author: Narsil
date: '2024-07-08'
url: https://github.com/Dao-AILab/flash-attention/pull/1025
source_category: upstream-code
architectures:
- sm100
tags:
- gemm
techniques: []
hardware_features: []
kernel_types:
- attention
- gemm
languages: []
captured_at: '2026-05-16'
status: merged
merge_sha: 8f873cc6acac2933d757b2ed6069518d619b341b
inclusion_reason: Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.
changed_paths:
- csrc/flash_attn/flash_api.cpp
- csrc/flash_attn/src/flash.h
- csrc/flash_attn/src/flash_fwd_kernel.h
- csrc/flash_attn/src/flash_fwd_launch_template.h
- csrc/flash_attn/src/static_switch.h
- flash_attn/flash_attn_interface.py
- setup.py
- tests/test_flash_attn.py
artifact_dir: evidence/pull-bundles/flash-attention/gh-1025
head_ref_oid: c0fa65c8831eae3249cda7c3d9c9d548f758a03b
---
# [WIP] First draft for softcapping.

- Repository: `Dao-AILab/flash-attention`
- PR: [1025](https://github.com/Dao-AILab/flash-attention/pull/1025)
- Decision: `Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.`
- Reason: Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.

## Changed Paths

- `csrc/flash_attn/flash_api.cpp`
- `csrc/flash_attn/src/flash.h`
- `csrc/flash_attn/src/flash_fwd_kernel.h`
- `csrc/flash_attn/src/flash_fwd_launch_template.h`
- `csrc/flash_attn/src/static_switch.h`
- `flash_attn/flash_attn_interface.py`
- `setup.py`
- `tests/test_flash_attn.py`

## Evidence Bundle

`evidence/pull-bundles/flash-attention/gh-1025`
