Skip to content

GitLab

Commit 1f21b69e authored by Toby Boyd's avatar Toby Boyd
Browse files

forced nccl has same num_packs as default.

parent bd86e960
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
official/utils/misc/distribution_utils.py
View file @ 1f21b69e
...@@ -42,7 +42,7 @@ def get_distribution_strategy(num_gpus, all_reduce_alg=None): ...@@ -42,7 +42,7 @@ def get_distribution_strategy(num_gpus, all_reduce_alg=None):
return tf.contrib.distribute.MirroredStrategy( return tf.contrib.distribute.MirroredStrategy(
num_gpus=num_gpus, num_gpus=num_gpus,
cross_tower_ops=tf.contrib.distribute.AllReduceCrossTowerOps( cross_tower_ops=tf.contrib.distribute.AllReduceCrossTowerOps(
all_reduce_alg, num_packs=num_gpus)) all_reduce_alg, num_packs=2))
else: else:
return tf.contrib.distribute.MirroredStrategy(num_gpus=num_gpus) return tf.contrib.distribute.MirroredStrategy(num_gpus=num_gpus)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment