ZeRO-2 (#217)

Updates for ZeRO stage 2 + ZeRO stage 1 w. RS Co-authored-by: Tunji Ruwase <olruwase@microsoft.com> Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com> Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com> Co-authored-by: Elton Zheng <eltonz@microsoft.com> Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com> Co-authored-by: yuxionghe <yuxhe@microsoft.com> Co-authored-by: Arash Ashari <arashari@microsoft.com>

ZeRO-2 (#217)
Updates for ZeRO stage 2 + ZeRO stage 1 w. RS Co-authored-by: Tunji Ruwase <olruwase@microsoft.com> Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com> Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com> Co-authored-by: Elton Zheng <eltonz@microsoft.com> Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com> Co-authored-by: yuxionghe <yuxhe@microsoft.com> Co-authored-by: Arash Ashari <arashari@microsoft.com>
f2ac7eaf · Jeff Rasley · GitHub · c61e23b4 · f2ac7eaf · f2ac7eaf
Unverified Commit f2ac7eaf authored May 19, 2020 by Jeff Rasley Committed by GitHub May 19, 2020
5 changed files
--- a/tests/unit/common.py
+++ b/tests/unit/common.py
--- a/tests/unit/test_checkpointing.py
+++ b/tests/unit/test_checkpointing.py
--- a/tests/unit/test_ds_config.py
+++ b/tests/unit/test_ds_config.py
--- a/tests/unit/test_fp16.py
+++ b/tests/unit/test_fp16.py
--- a/tests/unit/test_multi_output_model.py
+++ b/tests/unit/test_multi_output_model.py