"doc/vscode:/vscode.git/clone" did not exist on "8ea84de6cfb1eebff51085ae988e262ff9e28b36"
Commit 7025a092 authored by Dan Fu's avatar Dan Fu
Browse files

Make png images into jpg for dark mode

parent 4decc3c1
......@@ -40,14 +40,14 @@ Our graphs show sequence lengths between 128 and 4096 (when standard attention r
#### Speedup
![FlashAttention speedup](images/flashattn_speedup.png)
![FlashAttention speedup](images/flashattn_speedup.jpg)
We generally see 2-4X speedup at sequence lengths between 128 and 4K, and we see more speedup when using dropout and masking, since we fuse the kernels.
At sequence lengths that are popular with language models like 512 and 1K, we see speedups up to 4X when using dropout and masking.
#### Memory
![FlashAttention memory](images/flashattn_memory.png)
![FlashAttention memory](images/flashattn_memory.jpg)
We show memory savings in this graph (note that memory footprint is the same no matter if you use dropout or masking).
Memory savings are proportional to sequence length -- since standard attention has memory quadratic in sequence length, whereas FlashAttention has memory linear in sequence length.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment