Skip to content

GitLab

Unverified Commit bc4bbd9f authored by Younes Belkada's avatar Younes Belkada Committed by GitHub
Browse files

[`FA-2` / `Mistral`] Supprot fa-2 + right padding + forward (#26912) 

supprot fa-2 + right padding + forward
parent cbd278f0
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
src/transformers/models/mistral/modeling_mistral.py
View file @ bc4bbd9f
......@@ -879,6 +879,7 @@ class MistralModel(MistralPreTrainedModel):
padding_mask is not None
and hasattr(self.config, "_flash_attn_2_enabled")
and self.config._flash_attn_2_enabled
and past_key_values is not None
):
is_padding_right = padding_mask[:, -1].sum().item() != batch_size
if is_padding_right:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment