GitLab

Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>

* Push to remote

* Correctly handle multi output models by doing loss scaling in backward()
Unit tests for multi output models

* Fix formatting issues

* Formatting issues fix

* Fix formatting

* Update DeepSpeedExamples submodule
Enable Megatron model tests

* added zero_allow_untested_optimizer flag helpers

* add zero_allow_untested_optimizer config constants

* zero_allow_untested_optimizer logic with assertion

* Added unit test and CustomOptimizer helper class

* remove session_params in deepspeed_constants.py

* add constants info at README.md

* Better config filename

* Clean up configuration ToC

* fix docs permalink

* fix docs permalink

* Add coming soon to posts

* Add what's new section to main page

GitHub created a CNAME for us automatically. Cool.

* add support for torch 1.3+ builds inside a docker build environment
* remove apex imports

* allow installing a specific apex commit

* Enhancement: Ability to load checkpoint without loading the optimizer states. Unittest testing saving and loading checkpoint with fused, unfused and zero optimizer. The unitest takes about 165s

* add tests cases for onecycle policy with fp16/zero

* Make lr schedulers support fp16 optimizers

* Fix formatting

* More specific naming
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* Add --no_sudo to run without sudo

* Add --pip_mirror to set the pip mirror

* Default to running pip without sudo

* Typo

* Add --pip_sudo to Dockerfile and azure-pipelines.yml
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* add support for deepspeed env file to pass custom env values

* simplify deepspeed config example

* add text about mpirun

* add mpirun support for openmpi 4.0

* add master addr support from args

* switch mpi detection to use mpi4py

* set constant for default distributed port

* Make sure deepspeed_mpi exits in args