Merge pull request #313 from philipturner/patch-1

Metal FlashAttention

Merge pull request #313 from philipturner/patch-1
Metal FlashAttention
01c40dac · Tri Dao · GitHub · 72ad03ea · 4dbcaa14 · 01c40dac
Unverified Commit 01c40dac authored Jul 15, 2023 by Tri Dao Committed by GitHub Jul 15, 2023
Hide whitespace changes
Inline Side-by-side

Showing with 2 additions and 0 deletions

usage.md usage.md +2 -0

No files found.
--- a/usage.md
+++ b/usage.md
@@ -123,3 +123,5 @@ yields the fastest BERT training on cloud instances in MLPerf training 2.0 (June

 - [Jax](https://github.com/google/jax): an [implementation](https://github.com/lucidrains/flash-attention-jax)
  in Jax by [lucidrains](https://github.com/lucidrains/).
+
+- [Metal](https://developer.apple.com/metal): an [implementation](https://github.com/philipturner/metal-flash-attention) in Metal by Philip Turner. This ports FlashAttention to mobile GPU architectures such as Apple silicon.