Skip to content

GitLab

"deploy/vscode:/vscode.git/clone" did not exist on "15bf830f571a109f4096163fd39c5ef5c82d85a4"
Unverified Commit 01c40dac authored by Tri Dao's avatar Tri Dao Committed by GitHub
Browse files

Merge pull request #313 from philipturner/patch-1 

Metal FlashAttention
parents 72ad03ea 4dbcaa14
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
usage.md
View file @ 01c40dac
...@@ -123,3 +123,5 @@ yields the fastest BERT training on cloud instances in MLPerf training 2.0 (June ...@@ -123,3 +123,5 @@ yields the fastest BERT training on cloud instances in MLPerf training 2.0 (June
- [Jax](https://github.com/google/jax): an [implementation](https://github.com/lucidrains/flash-attention-jax) - [Jax](https://github.com/google/jax): an [implementation](https://github.com/lucidrains/flash-attention-jax)
in Jax by [lucidrains](https://github.com/lucidrains/). in Jax by [lucidrains](https://github.com/lucidrains/).
- [Metal](https://developer.apple.com/metal): an [implementation](https://github.com/philipturner/metal-flash-attention) in Metal by Philip Turner. This ports FlashAttention to mobile GPU architectures such as Apple silicon.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment