GitLab

* Push to remote

* Correctly handle multi output models by doing loss scaling in backward()
Unit tests for multi output models

* Fix formatting issues

* Formatting issues fix

* Fix formatting

* Update DeepSpeedExamples submodule
Enable Megatron model tests

* Enhancement: Ability to load checkpoint without loading the optimizer states. Unittest testing saving and loading checkpoint with fused, unfused and zero optimizer. The unitest takes about 165s

* Support legacy optimizer fusion as config option

* Configure for legacy optimizer fusion

* Update configuration jsons for new apex

* Porting BingBertSquad test

* Updating default paths.

* Enable model tests.

* Updating DeepSpeedExamples submodule

* Adding BingBertSquad's log uploads.

* Messed up the submodule again :-)

* simplifying the batch config, using a single assert to test for validity and allowing for specifying only the micro batch size

* Simplifying Batch Config, Adding ability to specify batch using just micro_batch, and adding a bunch of unit tests

* ran formatting

* Typo fixes and added the config file

* reformatting

* path fixes

* removing print statements