Skip to content

GitLab

Commit f2d8d410 authored by Tri Dao's avatar Tri Dao
Browse files

Edit README: support Turing (SM75)

parent d380e87f
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 3 deletions
README.md
View file @ f2d8d410
...@@ -9,7 +9,7 @@ Paper: https://arxiv.org/abs/2205.14135 ...@@ -9,7 +9,7 @@ Paper: https://arxiv.org/abs/2205.14135
## Alpha release (0.1). ## Alpha release (0.1).
To compile (requiring CUDA 11, NVCC, and an Ampere GPU): To compile (requiring CUDA 11, NVCC, and an Turing or Ampere GPU):
``` ```
python setup.py install python setup.py install
``` ```
...@@ -22,7 +22,7 @@ PYTHONPATH=$PWD python benchmarks/benchmark_flash_attention.py ...@@ -22,7 +22,7 @@ PYTHONPATH=$PWD python benchmarks/benchmark_flash_attention.py
``` ```
FlashAttention currently supports: FlashAttention currently supports:
1. Ampere GPUs (e.g., A100, RTX 3090). 1. Turing or Ampere GPUs (e.g., A100, RTX 3090, T4, RTX 2080).
2. fp16. 2. fp16.
3. Head dimensions 16, 32, 64. 3. Head dimensions 16, 32, 64.
...@@ -30,7 +30,7 @@ Our tentative roadmap: ...@@ -30,7 +30,7 @@ Our tentative roadmap:
1. [Jun 2022] Make package pip-installable. 1. [Jun 2022] Make package pip-installable.
2. ~~[Jun 2022] Support SM86 GPUs (e.g., RTX 3080, 3090)~~[Done]. 2. ~~[Jun 2022] Support SM86 GPUs (e.g., RTX 3080, 3090)~~[Done].
3. [Jun 2022] Refactor to use Cutlass. 3. [Jun 2022] Refactor to use Cutlass.
4. [Jun 2022] Support SM75 GPUs (e.g. T4). 4. ~~[Jun 2022] Support SM75 GPUs (e.g. T4)~~[Done].
5. [Jun 2022] Support bf16. 5. [Jun 2022] Support bf16.
6. [Jul 2022] Support head dimension 128. 6. [Jul 2022] Support head dimension 128.
7. [Jul 2022] Support SM70 GPUs (V100). 7. [Jul 2022] Support SM70 GPUs (V100).
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment