tests/model/Megatron_GPT2/run_checkpoint_test.py · 7ccc9daf9ec909b71c7261f8e77876774b1633af · OpenDAS / deepspeed

Support loading and saving ZeRO checkpoints with changing DP degree (#240) · 7ccc9daf

Olatunji Ruwase authored Jul 14, 2020



* Support saving and loading ZeRO checkpoints on different data
parallelism degree.

* Fix formatting

* Support checkpoint with varying GPU count in ZeRO stage 1

* Fix formatting

* Formatting fixes

* Update model tests

* Remove pprint

* Minor fix

* Fix formatting

* Update model tests
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

7ccc9daf

run_checkpoint_test.py 13.8 KB

Replace run_checkpoint_test.py