Skip to content

GitLab

"ml/git@developer.sourcefind.cn:OpenDAS/ollama.git" did not exist on "6600bd7d91deb07bec5832790168870c3180ccae"
Unverified Commit 886ef35c authored by Lysandre Debut's avatar Lysandre Debut Committed by GitHub
Browse files

Fix LXMERT with DataParallel (#7471)

parent 35e94c68
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
src/transformers/modeling_lxmert.py
View file @ 886ef35c
...@@ -958,7 +958,7 @@ class LxmertModel(LxmertPreTrainedModel): ...@@ -958,7 +958,7 @@ class LxmertModel(LxmertPreTrainedModel):
# positions we want to attend and -10000.0 for masked positions. # positions we want to attend and -10000.0 for masked positions.
# Since we are adding it to the raw scores before the softmax, this is # Since we are adding it to the raw scores before the softmax, this is
# effectively the same as removing these entirely. # effectively the same as removing these entirely.
extended_attention_mask = extended_attention_mask.to(dtype=next(self.parameters()).dtype) extended_attention_mask = extended_attention_mask.to(dtype=self.dtype)
extended_attention_mask = (1.0 - extended_attention_mask) * -10000.0 extended_attention_mask = (1.0 - extended_attention_mask) * -10000.0
# Process the visual attention mask # Process the visual attention mask
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment