Correctly zero padding index in TransformerSentenceEncoder

Summary: Pull Request resolved: https://github.com/pytorch/fairseq/pull/912

Differential Revision: D16536561

Pulled By: myleott

fbshipit-source-id: 54c5c20a826a14f4e690770e027bcb282acdf911

fairseq/modules/transformer_sentence_encoder.py

@@ -38,6 +38,7 @@ def init_bert_params(module):
             module.bias.data.zero_()
     if isinstance(module, nn.Embedding):
         module.weight.data.normal_(mean=0.0, std=0.02)
+        module.weight.data[module.padding_idx].zero_()
     if isinstance(module, MultiheadAttention):
         module.in_proj_weight.data.normal_(mean=0.0, std=0.02)