GitLab

* add tutorials

* add new context, modify and delete existing docs

* remove duplicate labels

* modify layout and more nits

* address comments

* fix merge conflicts

* modify doc string

* add offload docs

* add tutorial

* remove print

* remove print statement

* modify import

* modify constants

* modify README and add Offload symbol

* fix lint

* smaller mods

* lint errors

* Update README.md

added the references at the bottom of the readme

* address comments

* doc changes

* add blank line
Co-authored-by: Anjali Sridhar <anj@devfair0443.h2.fair>
Co-authored-by: Vittorio Caggiano <caggiano@gmail.com>

* fix broken link

* remove link

* add direct link

* add direct link

* add direct link

* add direct link

* add direct link

* add direct link
Co-authored-by: Anjali Sridhar <anj@devfair0443.h2.fair>

* [doc]: AdaScale example and notes

* formatted notes correctly as suggested by Benjamin

* added feature and unit test to make sure lr_scheduler works

* update the example with lr_scheduler

* fixed doc with "make html"

* addressed Mike's suggestions

Thanks Jessica for the heads up !

* adding a shard-aware GradScaler wrap, credits to Sean Naren for the idea
* adding stubs & explanations in the documentation

- removed experimental warning as we have validated it on cifar and
imagenet, transformer is looking good so far too.
- fixed API doc formatting
- make it consistent with the other code in the repo
- tested by making the doc locally and inspect the results

* wip_example

* [wip]mnist_pipe_example

* [wip]mnist_pipe_example

* [wip]mnist_pipe_example

* [wip]mnist_pipe_example

* [wip]mnist_oss_example

* working prototype

* added tutorial script

* update tutorial

* Update mnist_test_oss.py

* Update mnist_test_oss.py

* Update mnist_test_oss.py

* Update mnist_test_pipe.py

* Update tutorial_oss.py

* Update tutorial_pipe.py

* Update tutorial_pipe.py

* Update mnist_test_oss.py

* Update tutorial_pipe.py

* Update mnist_test_pipe.py

* Update tutorial_pipe.py

* fix black

* fix flacke8

* general fixes

* add example oss+pipe

* fix isort

* Update mnist_test_pipe.py

* fix black
Co-authored-by: Vittorio Caggiano <caggiano@devfair0253.h2.fair>

* fixing the readme for oss

* Add implementation of AdaScale

* add adascale docs

* bugfix
* adjust default non-regression loss, not all_reduced now

* wrapping the model in DDP in the tutorial

* typo

* Doc extensions to some APIs
* FIx the benchmark and tutorial

Add a small tutorial, similar to the OSS README