GitLab

* Add reducer class in parallel/distributed.

* Separate DDP and Reducer examples.

* Don't confuse DDP and reducer in example.

Handle set/get state for DDP, remove stream which cant be pickled.

* Proper implementation of LARC clipping
 * Documentation of LARC class
 * Modification of FP16_Optimizer to absorb optimizer instance that's being wrapped instead of creating new optimizer instance of same class.

Fixes to validation in imagenet example scripts.

Fixes to validation in imagenet example scripts. Precision and loss reporting modified to be consistent with train.

Fix to imagenet main.py data normalization.

* More stringent check for distributed refresh

Fixed deadlock issue