Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
apex
Commits
e12c1ec3
Commit
e12c1ec3
authored
Oct 10, 2018
by
Michael Carilli
Browse files
Docstring updates
parent
8add2b07
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
3 deletions
+2
-3
apex/parallel/distributed.py
apex/parallel/distributed.py
+2
-3
No files found.
apex/parallel/distributed.py
View file @
e12c1ec3
...
...
@@ -64,7 +64,7 @@ class Reducer(object):
:class:`Reducer` is designed to work with the launch utility script
``apex.parallel.multiproc.py`` or the upstream launch utility script
``torch.distributed.launch`` with --nproc_per_node <=
the
number of gpus per node.
``torch.distributed.launch`` with
``
--nproc_per_node <= number of gpus per node
``
.
For forward compatibility, ``torch.distributed.launch`` is recommended.
When used with these launchers, :class:`Reducer` assumes 1:1 mapping of processes to GPUs.
It also assumes that your script calls ``torch.cuda.set_device(args.rank)`` before creating the model.
...
...
@@ -105,11 +105,10 @@ class DistributedDataParallel(Module):
:class:`DistributedDataParallel` is designed to work with the launch utility script
``apex.parallel.multiproc.py`` or the upstream launch utility script
``torch.distributed.launch`` with --nproc_per_node <=
the
number of gpus per node.
``torch.distributed.launch`` with
``
--nproc_per_node <= number of gpus per node
``
.
For forward compatibility, ``torch.distributed.launch`` is recommended.
When used with these launchers, :class:`DistributedDataParallel` assumes 1:1 mapping of processes to GPUs.
It also assumes that your script calls ``torch.cuda.set_device(args.rank)`` before creating the model.
When used with these launchers, :class:`DistributedDataParallel` assumes 1:1 mapping of processes to GPUs.
https://github.com/NVIDIA/apex/tree/master/examples/distributed shows detailed usage.
https://github.com/NVIDIA/apex/tree/master/examples/imagenet shows another example
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment