Enable all_reduce_alg flag

PiperOrigin-RevId: 292029030

Enable all_reduce_alg flag
PiperOrigin-RevId: 292029030
4f3acbda · Zongwei Zhou · A. Unique TensorFlower · dbdd17c4 · 4f3acbda
Commit 4f3acbda authored Jan 28, 2020 by Zongwei Zhou Committed by A. Unique TensorFlower Jan 28, 2020
Show whitespace changes
Inline Side-by-side

Showing with 2 additions and 2 deletions

official/nlp/bert/common_flags.py official/nlp/bert/common_flags.py +2 -2

No files found.
--- a/official/nlp/bert/common_flags.py
+++ b/official/nlp/bert/common_flags.py
@@ -71,7 +71,7 @@ def define_common_bert_flags():
      'Specifies the type of the model. '
      'If "bert", will use canonical BERT; if "albert", will use ALBERT model.')

-  # Adds flags for mixed precision training.
+  # Adds flags for mixed precision and multi-worker training.
  flags_core.define_performance(
      num_parallel_calls=False,
      inter_op=False,
@@ -81,7 +81,7 @@ def define_common_bert_flags():
      dtype=True,
      dynamic_loss_scale=True,
      loss_scale=True,
-      all_reduce_alg=False,
+      all_reduce_alg=True,
      num_packs=False,
      enable_xla=True,
      fp16_implementation=True,