GitLab

* add text about mpirun

* add mpirun support for openmpi 4.0

* add master addr support from args

* switch mpi detection to use mpi4py

* set constant for default distributed port

* Make sure deepspeed_mpi exits in args

* add auto-detect to torch dist init

* update tests to infer distributed init status

* prevent crash if dist_init_required is True but already initiliazed

* only init if safe to do so (forgot to add this file in prev commit)

* Update scripts to handle cases where you have other VMs in your sub

* Support subs with other VMs and fix for PDSH permission error

* Minor fix to support subs with other VMs

* Added shutdown with or without delete VM option

In Azure deallocate is like machine shutdown (and prevents billing). You can restart deallocated VM. To fully drop the VM delete is used. This command with "-d" option will fully delete the VM. Without any argument it justs deallocates / shutd down the VM.

* Update scripts to handle cases where you have other VMs in your sub

* Support subs with other VMs and fix for PDSH permission error

* Minor fix to support subs with other VMs

* Removes DeepSpeedDataSource

* dropping unused imports
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* Support legacy optimizer fusion as config option

* Configure for legacy optimizer fusion

* Update configuration jsons for new apex

Also a fix for #94 

Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>

* add install requirements command line

* add pillow library to fix version

* modify to uppercase
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

bug fixes for adamw/lamb and corresponding tests

* Porting BingBertSquad test

* Updating default paths.

* Enable model tests.

* Updating DeepSpeedExamples submodule

* Adding BingBertSquad's log uploads.

* Messed up the submodule again :-)

* Set up CI with Azure Pipelines for docker build/push

* bump tf version in dockerfile

* Update install.sh

* Fix broken link for the 1Cycle doc.

* Removed the 1Cycle link from README.md.

* Importing 1Cycle tutorial.

* image paths

* Added LR schedule figure

* line wrap

* lowercase name

* Updating README links

* typo

Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* Increasing section headers

* Move testing under contributing
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

Bumps [tensorflow-gpu](https://github.com/tensorflow/tensorflow) from 1.14.0 to 1.15.2.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v1.14.0...v1.15.2

)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

Add Azure tutorial text and scripts