GitLab

* [zero] polish sharded param name

* polish code

* polish

* polish code

* polish

* polsih

* polish

* polish code

* shard strategy receive pg in shard() / gather()

* update zero engine

* polish code

* Revert "polish code"

This reverts commit 8cf7ff08.

* Revert "rename variables"

This reverts commit e99af94a.

* Revert "remove surplus imports"

This reverts commit 46add4a5.

* Revert "update sharded optim and fix zero init ctx"

This reverts commit 57567ee7.

* Added tensor detector

* Added the - states

* Allowed change include_cpu when detect()

* place params on cpu after zero init context

* polish code

* bucketzed cpu gpu tensor transter

* find a bug in sharded optim unittest

* add offload unittest for ShardedOptimV2.

* polish code and make it more robust

* refactored test with component func

* fixed bug

* Added activation offload

* Fixed the import bug, used the pytest

This reverts commit bef05489b642385c80e59fe757d598efd1752ecf.

* place params on cpu after zero init context

* polish code