GitLab

* [zero] add strict ddp mode for chunk init

* [gemini] update gpt example

* add chunk manager init function

* fix unit tests

* add comment

* add flush=True

This reverts commit 5be118f4.

* process group supports getting ranks in group

* chunk mgr receives a process group

* update unit test

* fix unit tests

* ColoDDP supports overwriting default process group

* rename ColoDDPV2 to ZeroDDP

* add docstr for ZeroDDP

* polish docstr

* add save/load state dict for ColoDDP

* add unit test

* refactor unit test folder

* polish unit test

* rename unit test

* add set_params_to_ignore for ColoDDP

* polish code

* fix zero hook v2

* add unit test

* polish docstr