Unverified Commit 9acce7de authored by Younes Belkada's avatar Younes Belkada Committed by GitHub
Browse files

Core: Fix copies on main (#29624)

fix fix copies
parent be3fd8a2
...@@ -454,7 +454,7 @@ class GPTJFlashAttention2(GPTJAttention): ...@@ -454,7 +454,7 @@ class GPTJFlashAttention2(GPTJAttention):
attention_mask (`torch.Tensor`): attention_mask (`torch.Tensor`):
The padding mask - corresponds to a tensor of size `(batch_size, seq_len)` where 0 stands for the The padding mask - corresponds to a tensor of size `(batch_size, seq_len)` where 0 stands for the
position of padding tokens and 1 for the position of non-padding tokens. position of padding tokens and 1 for the position of non-padding tokens.
dropout (`int`, *optional*): dropout (`float`):
Attention dropout Attention dropout
softmax_scale (`float`, *optional*): softmax_scale (`float`, *optional*):
The scaling of QK^T before applying softmax. Default to 1 / sqrt(head_dim) The scaling of QK^T before applying softmax. Default to 1 / sqrt(head_dim)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment