GitLab

* beautiful

* IT'S WORKING

* Hopefully fix race condition for fallback hook

* Updating test

* shared_param -> delayed_allreduce

* Adding a safety check

* One more check

* syntax...

* Fix appears to work in Tomasz's example.

* Somehow shared_param got de-enabled again?

* Remove some fp16 examples that don't converge

Default static loss scale of 1.0 (default value) for resnet50 doesn't converge. Either remove example or put static loss scale 128 on it, which is known to converge well.

* Update README.md

* WIP: update to support new RNN backend code

* small refactor

* add test for rnn w/ packed sequences

amp unit tests

* Add reducer class in parallel/distributed.

* Separate DDP and Reducer examples.

* Don't confuse DDP and reducer in example.