Remove extraneous call to RNG in multi-GPU code path

Summary: Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/865

Differential Revision: D17510276

Pulled By: myleott

fbshipit-source-id: 24119402ad5fe95a1312fadb77bafe49a9197c6b

fairseq/distributed_utils.py

@@ -87,7 +87,7 @@ def distributed_init(args):
             socket.gethostname(), args.distributed_rank), flush=True)
 
         # perform a dummy all-reduce to initialize the NCCL communicator
-        dist.all_reduce(torch.rand(1).cuda())
+        dist.all_reduce(torch.zeros(1).cuda())
 
         suppress_output(is_master(args))