Skip to content

GitLab

Commit 1bbebccc authored by Tri Dao's avatar Tri Dao
Browse files

Edit README to mention bf16 support

parent de19de7a
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
README.md
View file @ 1bbebccc
...@@ -23,8 +23,8 @@ PYTHONPATH=$PWD python benchmarks/benchmark_flash_attention.py ...@@ -23,8 +23,8 @@ PYTHONPATH=$PWD python benchmarks/benchmark_flash_attention.py
FlashAttention currently supports: FlashAttention currently supports:
1. Turing or Ampere GPUs (e.g., A100, RTX 3090, T4, RTX 2080). 1. Turing or Ampere GPUs (e.g., A100, RTX 3090, T4, RTX 2080).
2. fp16. 2. fp16 and bf16 (bf16 requires Ampere GPUs).
3. Head dimensions 16, 32, 64, 128 (bwd requires A100). 3. Head dimensions 16, 32, 64, 128 (head dim 128 backward requires A100).
Our tentative roadmap: Our tentative roadmap:
1. [Jun 2022] Make package pip-installable. 1. [Jun 2022] Make package pip-installable.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment