GitLab

* sharded model supports reuse fp16 shard

* rename variable

* polish code

* polish code

* polish code

* sharded optim support hybrid cpu adam

* update unit test

* polish docstring

* [zero] polish sharded param name

* polish code

* polish

* polish code

* polish

* polsih

* polish

Co-authored-by: github-actions <github-actions@github.com>

* [readme] add experimental visualisation regarding ColossalAI with ZeRO (#476)

* Hotfix/readme (#478)

* add experimental visualisation regarding ColossalAI with ZeRO

* adjust newly-added figure size

* [test] added rerun on exception function

* polish code

* update rst

* remove empty rst

* polish sharded model docstr

* polish sharded optim docstr

* polish zero docstr

* polish shard strategy docstr

* polish code

* shard strategy receive pg in shard() / gather()

* update zero engine

* polish code

* Revert "polish code"

This reverts commit 8cf7ff08.

* Revert "rename variables"

This reverts commit e99af94a.

* Revert "remove surplus imports"

This reverts commit 46add4a5.

* Revert "update sharded optim and fix zero init ctx"

This reverts commit 57567ee7.