GitLab

1) CSR parameter names should end with .weight. 
2) When using basic optimizer directly, DeepSpeed should handle zero_grad. Letting the basic optimizer do the zero_grad resulted in residual gradients in the embedding layer due to unknown reasons.

Remove explicit torch version requirement so that we can more easily support other versions

Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>

* Push to remote

* Correctly handle multi output models by doing loss scaling in backward()
Unit tests for multi output models

* Fix formatting issues

* Formatting issues fix

* Fix formatting

* Update DeepSpeedExamples submodule
Enable Megatron model tests

* added zero_allow_untested_optimizer flag helpers

* add zero_allow_untested_optimizer config constants

* zero_allow_untested_optimizer logic with assertion

* Added unit test and CustomOptimizer helper class

* remove session_params in deepspeed_constants.py

* add constants info at README.md

* Better config filename

* Clean up configuration ToC

* fix docs permalink

* fix docs permalink

* Add coming soon to posts

* Add what's new section to main page

GitHub created a CNAME for us automatically. Cool.

* add support for torch 1.3+ builds inside a docker build environment
* remove apex imports

* allow installing a specific apex commit

* Enhancement: Ability to load checkpoint without loading the optimizer states. Unittest testing saving and loading checkpoint with fused, unfused and zero optimizer. The unitest takes about 165s

* add tests cases for onecycle policy with fp16/zero

* Make lr schedulers support fp16 optimizers

* Fix formatting

* More specific naming
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* Add --no_sudo to run without sudo

* Add --pip_mirror to set the pip mirror

* Default to running pip without sudo

* Typo

* Add --pip_sudo to Dockerfile and azure-pipelines.yml
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>