Adding masked_lm_dictionary to pytorch_translate (#630)

Summary:
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/630

Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/629

Pull Request resolved: https://github.com/pytorch/translate/pull/562

Pull Request resolved: https://github.com/pytorch/fairseq/pull/774

forked masked_lm_dictionary from fairseq
changed import in pytorch_translate to use the new masked_lm_dictionary
registered cooresponding tasks

Reviewed By: liezl200

Differential Revision: D15410352

fbshipit-source-id: 06516caabdd4dc5cdee9ad1d8025978f4eea6c4b

fairseq/models/transformer_from_pretrained_xlm.py

@@ -44,13 +44,13 @@ class TransformerFromPretrainedXLMModel(TransformerModel):
         )
 
     @classmethod
-    def build_model(cls, args, task):
+    def build_model(self, args, task, cls_dictionary=MaskedLMDictionary):
         assert hasattr(args, "pretrained_xlm_checkpoint"), (
             "You must specify a path for --pretrained-xlm-checkpoint to use "
             "--arch transformer_from_pretrained_xlm"
         )
-        assert isinstance(task.source_dictionary, MaskedLMDictionary) and isinstance(
-            task.target_dictionary, MaskedLMDictionary
+        assert isinstance(task.source_dictionary, cls_dictionary) and isinstance(
+            task.target_dictionary, cls_dictionary
         ), (
             "You should use a MaskedLMDictionary when using --arch "
             "transformer_from_pretrained_xlm because the pretrained XLM model "