Commit adaa9137 authored by Michael Carilli's avatar Michael Carilli
Browse files

Readme updates + version in Sphinx

parent 0a092aaf
fp16_optimizer.py contains `FP16_Optimizer`, a Python class designed to wrap an existing Pytorch optimizer and automatically enable master parameters and loss scaling in a manner transparent to the user. To use `FP16_Optimizer`, only two lines of one's Python model need to change. fp16_optimizer.py contains `FP16_Optimizer`, a Python class designed to wrap an existing Pytorch optimizer and automatically enable master parameters and loss scaling in a manner transparent to the user. To use `FP16_Optimizer`, only two lines of one's Python model need to change.
### [FP16_Optimizer API documentation](https://nvidia.github.io/apex/fp16_utils.html#automatic-management-of-master-params-loss-scaling) #### [FP16_Optimizer API documentation](https://nvidia.github.io/apex/fp16_utils.html#automatic-management-of-master-params-loss-scaling)
### [Simple examples with FP16_Optimizer](https://github.com/NVIDIA/apex/tree/master/examples/FP16_Optimizer_simple) #### [Simple examples with FP16_Optimizer](https://github.com/NVIDIA/apex/tree/master/examples/FP16_Optimizer_simple)
### [Imagenet with FP16_Optimizer](https://github.com/NVIDIA/apex/tree/master/examples/imagenet) #### [Imagenet with FP16_Optimizer](https://github.com/NVIDIA/apex/tree/master/examples/imagenet)
### [word_language_model with FP16_Optimizer](https://github.com/NVIDIA/apex/tree/master/examples/word_language_model) #### [word_language_model with FP16_Optimizer](https://github.com/NVIDIA/apex/tree/master/examples/word_language_model)
fp16_util.py contains a number of utilities to manually manage master parameters and loss scaling, if the user chooses. fp16_util.py contains a number of utilities to manually manage master parameters and loss scaling, if the user chooses.
### [Manual management documentation](https://nvidia.github.io/apex/fp16_utils.html#manual-master-parameter-management) #### [Manual management documentation](https://nvidia.github.io/apex/fp16_utils.html#manual-master-parameter-management)
The [Imagenet with FP16_Optimizer](https://github.com/NVIDIA/apex/tree/master/examples/imagenet) and [word_language_model with FP16_Optimizer](https://github.com/NVIDIA/apex/tree/master/examples/word_language_model) directories also contain `main.py` files that demonstrate manual management of master parameters and static loss scaling. These examples illustrate what sort of operations `FP16_Optimizer` is performing automatically. The [Imagenet with FP16_Optimizer](https://github.com/NVIDIA/apex/tree/master/examples/imagenet) and [word_language_model with FP16_Optimizer](https://github.com/NVIDIA/apex/tree/master/examples/word_language_model) directories also contain `main.py` files that demonstrate manual management of master parameters and static loss scaling. These examples illustrate what sort of operations `FP16_Optimizer` is performing automatically.
...@@ -6,10 +6,10 @@ transfers required. ...@@ -6,10 +6,10 @@ transfers required.
multiproc.py contains the source code for `apex.parallel.multiproc`, a launch utility that places one process on each of the node's available GPUs. multiproc.py contains the source code for `apex.parallel.multiproc`, a launch utility that places one process on each of the node's available GPUs.
### [API Documentation](https://nvidia.github.io/apex/parallel.html) #### [API Documentation](https://nvidia.github.io/apex/parallel.html)
### [Example/Walkthrough](https://github.com/NVIDIA/apex/tree/master/examples/distributed) #### [Example/Walkthrough](https://github.com/NVIDIA/apex/tree/master/examples/distributed)
### [Imagenet Example w/Mixed Precision](https://github.com/NVIDIA/apex/tree/master/examples/imagenet) #### [Imagenet Example w/Mixed Precision](https://github.com/NVIDIA/apex/tree/master/examples/imagenet)
...@@ -73,10 +73,10 @@ author = 'Christian Sarofeen, Natalia Gimelshein, Michael Carilli, Raul Puri' ...@@ -73,10 +73,10 @@ author = 'Christian Sarofeen, Natalia Gimelshein, Michael Carilli, Raul Puri'
# The short X.Y version. # The short X.Y version.
# TODO: change to [:2] at v1.0 # TODO: change to [:2] at v1.0
# version = 'master (' + torch.__version__ + ' )' # version = 'master (' + torch.__version__ + ' )'
version = '0.0' version = '0.1'
# The full version, including alpha/beta/rc tags. # The full version, including alpha/beta/rc tags.
# TODO: verify this works as expected # TODO: verify this works as expected
release = '0.0.0' release = '0.1.0'
# The language for content autogenerated by Sphinx. Refer to documentation # The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages. # for a list of supported languages.
......
...@@ -5,7 +5,7 @@ half and float parameters, only two lines of your training script need to change ...@@ -5,7 +5,7 @@ half and float parameters, only two lines of your training script need to change
1. Construct an `FP16_Optimizer` instance from an existing optimizer. 1. Construct an `FP16_Optimizer` instance from an existing optimizer.
2. Replace `loss.backward()` with `optimizer.backward(loss)`. 2. Replace `loss.backward()` with `optimizer.backward(loss)`.
[Full API Documentation](https://nvidia.github.io/apex/fp16_utils.html#automatic-management-of-master-params-loss-scaling) #### [Full API Documentation](https://nvidia.github.io/apex/fp16_utils.html#automatic-management-of-master-params-loss-scaling)
See "Other Options" at the bottom of this page for some cases that require special treatment. See "Other Options" at the bottom of this page for some cases that require special treatment.
......
...@@ -13,11 +13,11 @@ global interpreter lock contention versus having a single process (with a single ...@@ -13,11 +13,11 @@ global interpreter lock contention versus having a single process (with a single
overlapping communication with computation during ``backward()`` and bucketing smaller gradient overlapping communication with computation during ``backward()`` and bucketing smaller gradient
transfers to reduce the total number of transfers required. transfers to reduce the total number of transfers required.
[API Documentation](https://nvidia.github.io/apex/parallel.html) #### [API Documentation](https://nvidia.github.io/apex/parallel.html)
[Source Code](https://github.com/NVIDIA/apex/tree/master/apex/parallel) #### [Source Code](https://github.com/NVIDIA/apex/tree/master/apex/parallel)
[Another Example: Imagenet with mixed precision](https://github.com/NVIDIA/apex/tree/master/examples/imagenet) #### [Another Example: Imagenet with mixed precision](https://github.com/NVIDIA/apex/tree/master/examples/imagenet)
## Getting started ## Getting started
Prior to running please run Prior to running please run
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment