Skip to content

GitLab

Unverified Commit 0ac38750 authored by Hugo Laurençon's avatar Hugo Laurençon Committed by GitHub
Browse files

Fix DeepSpeed issue with Idefics (#26393) 

Fix deepspeed issue with Idefics
parent 6ce6a5ad
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
src/transformers/models/idefics/modeling_idefics.py
View file @ 0ac38750
...@@ -428,7 +428,7 @@ class IdeficsDecoupledLinear(nn.Linear): ...@@ -428,7 +428,7 @@ class IdeficsDecoupledLinear(nn.Linear):
output = F.linear(input, self.weight, self.bias) output = F.linear(input, self.weight, self.bias)
if self.out_additional_features > 0: if self.out_additional_features > 0:
additional_features = F.linear(input, self.additional_fc.weight, self.additional_fc.bias) additional_features = self.additional_fc(input)
output = torch.cat((output, additional_features), -1) output = torch.cat((output, additional_features), -1)
return output return output
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment