GitLab

* Added tensor detector

* Added the - states

* Allowed change include_cpu when detect()

* place params on cpu after zero init context

* polish code

* bucketzed cpu gpu tensor transter

* find a bug in sharded optim unittest

* add offload unittest for ShardedOptimV2.

* polish code and make it more robust

* refactored test with component func

* fixed bug

* Added activation offload

* Fixed the import bug, used the pytest

This reverts commit bef05489b642385c80e59fe757d598efd1752ecf.

* place params on cpu after zero init context

* polish code

* add zero init context

* add more flags for zero init context
fix bug of repeated converting param to ShardedParamV2

* polish code

Fixed bug for learning rate scheduler

* Added CPU Adam

* finished the cpu adam

* updated the license

* delete useless parameters, removed resnet

* modified the method off cpu adam unittest

* deleted some useless codes

* removed useless codes
Co-authored-by: ver217 <lhx0217@gmail.com>
Co-authored-by: Frank Lee <somerlee.9@gmail.com>
Co-authored-by: jiaruifang <fangjiarui123@gmail.com>

* init shard param from shape tuple

* add more unitest for shard param

* add set_payload method for ShardedParam

* [zero] add shareded tensor class

* polish code

* add shard stratgy

* move shard and gather logic to shard strategy from shard tensor.

* polish code

* init shard param from shape tuple

* add more unitest for shard param

* add set_payload method for ShardedParam

* [zero] add shareded tensor class

* polish code

* init shard param from shape tuple

* add more unitest for shard param

* add more unittests to shareded param

* added unit test for sharded optimizer

* refactor for elegance