tests/model/Megatron_GPT2/run_checkpoint_test.py · 79093d74aa1632e5eac937480774b6fd29084228 · OpenDAS / deepspeed

"vscode:/vscode.git/clone" did not exist on "b3de3b988abf39c0ea8d75379922437da1062a3b"

Support loading and saving ZeRO checkpoints with changing DP degree (#240) · 7ccc9daf

Olatunji Ruwase authored Jul 14, 2020



* Support saving and loading ZeRO checkpoints on different data
parallelism degree.

* Fix formatting

* Support checkpoint with varying GPU count in ZeRO stage 1

* Fix formatting

* Formatting fixes

* Update model tests

* Remove pprint

* Minor fix

* Fix formatting

* Update model tests
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

7ccc9daf

run_checkpoint_test.py 13.8 KB

Replace run_checkpoint_test.py