Unverified Commit 41cb9097 authored by Kiarash Jamali's avatar Kiarash Jamali Committed by GitHub
Browse files

Change default dropout value in documentation

Documentation says default is 0.1, but the code has attention_dropout default at 0.0
parent d5098324
......@@ -16,7 +16,7 @@ class FlashAttention(nn.Module):
(default: 1/sqrt(d_keys) where d_keys is computed at
runtime)
attention_dropout: The dropout rate to apply to the attention
(default: 0.1)
(default: 0.0)
"""
def __init__(self, softmax_scale=None, attention_dropout=0.0, device=None, dtype=None):
super().__init__()
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment