---
id: pr-thunderkittens-112
repo: HazyResearch/ThunderKittens
pr: 112
title: Add Ring Attention Kernel
author: StuartSul
date: '2025-04-15'
url: https://github.com/HazyResearch/ThunderKittens/pull/112
source_category: upstream-code
architectures:
- sm100
tags:
- attention
- attention kernel
techniques: []
hardware_features: []
kernel_types:
- attention
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: d69697a3337e31d0060178c9049f1184e7e7ad7f
inclusion_reason: Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.
changed_paths:
- kernels/ring_attention/Makefile
- kernels/ring_attention/setup.py
- kernels/ring_attention/tk_ring_attention.cu
- tests/python/ring_attention/benchmark.py
- tests/python/ring_attention/implementations.py
- tests/python/ring_attention/original_ring_attention/__init__.py
- tests/python/ring_attention/original_ring_attention/ringattention_inference.py
- tests/python/ring_attention/original_ring_attention/ringattention_jax.py
- tests/python/ring_attention/original_ring_attention/ringattention_pallas_gpu.py
- tests/python/ring_attention/original_ring_attention/ringattention_pallas_tpu.py
- tests/python/ring_attention/test_correctness.py
artifact_dir: evidence/pull-bundles/thunderkittens/gh-112
head_ref_oid: 1506a7a89be97c3b9680ee50db5905f4a78c842e
---
# Add Ring Attention Kernel

- Repository: `HazyResearch/ThunderKittens`
- PR: [112](https://github.com/HazyResearch/ThunderKittens/pull/112)
- Decision: `Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.`
- Reason: Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.

## Changed Paths

- `kernels/ring_attention/Makefile`
- `kernels/ring_attention/setup.py`
- `kernels/ring_attention/tk_ring_attention.cu`
- `tests/python/ring_attention/benchmark.py`
- `tests/python/ring_attention/implementations.py`
- `tests/python/ring_attention/original_ring_attention/__init__.py`
- `tests/python/ring_attention/original_ring_attention/ringattention_inference.py`
- `tests/python/ring_attention/original_ring_attention/ringattention_jax.py`
- `tests/python/ring_attention/original_ring_attention/ringattention_pallas_gpu.py`
- `tests/python/ring_attention/original_ring_attention/ringattention_pallas_tpu.py`
- `tests/python/ring_attention/test_correctness.py`

## Evidence Bundle

`evidence/pull-bundles/thunderkittens/gh-112`
