Setting Global sync to 50 in BMUF

Summary:
In all our final settings, we are using global_sync = 50 and we get comparable results with DDP and caffe2.

Setting the default global-sync-iter = 50
and users can just define --use-bmuf to enable it for training.

Reviewed By: skritika

Differential Revision: D17765094

fbshipit-source-id: 369591eeff266d757f89e1fc8dda01711146fdbc

fairseq/optim/bmuf.py

@@ -47,7 +47,7 @@ class FairseqBMUF(FairseqOptimizer):
         )
         parser.add_argument(
             "--global-sync-iter",
-            default=10,
+            default=50,
             type=int,
             help="Iteration for syncing global model",
         )