Skip to content

GitLab

Unverified Commit c0660df5 authored by Thomas Wolf's avatar Thomas Wolf Committed by GitHub
Browse files

Merge pull request #357 from pglock/feature/354-use-dropout-layer-gpt 

Use Dropout Layer in OpenAIGPTMultipleChoiceHead
parents 7cc35c31 6190e8ce
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
pytorch_pretrained_bert/modeling_openai.py
View file @ c0660df5
...@@ -374,6 +374,7 @@ class OpenAIGPTMultipleChoiceHead(nn.Module): ...@@ -374,6 +374,7 @@ class OpenAIGPTMultipleChoiceHead(nn.Module):
# (bsz, num_choices, 1, hidden_size) # (bsz, num_choices, 1, hidden_size)
multiple_choice_h = hidden_states.gather(2, mc_token_ids).squeeze(2) multiple_choice_h = hidden_states.gather(2, mc_token_ids).squeeze(2)
# (bsz, num_choices, hidden_size) # (bsz, num_choices, hidden_size)
multiple_choice_h = self.dropout(multiple_choice_h.transpose(1, 2)).transpose(1, 2)
multiple_choice_logits = self.linear(multiple_choice_h).squeeze(-1) multiple_choice_logits = self.linear(multiple_choice_h).squeeze(-1)
# (bsz, num_choices) # (bsz, num_choices)
return multiple_choice_logits return multiple_choice_logits
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment