Unverified Commit 3f52c685 authored by Ibraheem Moosa's avatar Ibraheem Moosa Committed by GitHub
Browse files

Fix classifier dropout in AlbertForMultipleChoice (#13087)

Classification head of AlbertForMultipleChoice uses `hidden_dropout_prob` instead of `classifier_dropout_prob`.  This
is not desirable as we cannot change classifer head dropout probability without changing the dropout probabilities of
the whole model.
parent c89180a9
...@@ -1282,7 +1282,7 @@ class AlbertForMultipleChoice(AlbertPreTrainedModel): ...@@ -1282,7 +1282,7 @@ class AlbertForMultipleChoice(AlbertPreTrainedModel):
super().__init__(config) super().__init__(config)
self.albert = AlbertModel(config) self.albert = AlbertModel(config)
self.dropout = nn.Dropout(config.hidden_dropout_prob) self.dropout = nn.Dropout(config.classifier_dropout_prob)
self.classifier = nn.Linear(config.hidden_size, 1) self.classifier = nn.Linear(config.hidden_size, 1)
self.init_weights() self.init_weights()
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment