GitLab

* [doc]: AdaScale example and notes

* formatted notes correctly as suggested by Benjamin

* added feature and unit test to make sure lr_scheduler works

* update the example with lr_scheduler

* fixed doc with "make html"

* addressed Mike's suggestions

- removed experimental warning as we have validated it on cifar and
imagenet, transformer is looking good so far too.
- fixed API doc formatting
- make it consistent with the other code in the repo
- tested by making the doc locally and inspect the results

* Add implementation of AdaScale

* add adascale docs

* bugfix
* adjust default non-regression loss, not all_reduced now

* wrapping the model in DDP in the tutorial

* typo

* Doc extensions to some APIs
* FIx the benchmark and tutorial

Add a small tutorial, similar to the OSS README