GitLab

* reorgnize colotensor directory

* polish code

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* refactor ppschedule to support tensor list

* polish

* add zero optimizer

* torch ok

* unit test ok

* polish code

* fix bugs

* polish unit test

* polish zero optim

* polish colo ddp v2

* refactor folder structure

* add comment

* polish unit test

* polish zero optim

* polish unit test

Co-authored-by: github-actions <github-actions@github.com>

* impl chunk manager

* impl param op hook

* add reduce_chunk

* add zero hook v2

* add zero dp

* fix TensorInfo

* impl load balancing when using zero without chunk

* fix zero hook

* polish chunk

* fix bugs

* ddp ok

* zero ok

* polish code

* fix bugs about load balancing

* polish code

* polish code

* add ene-to-end test

* polish code

* polish code

* polish code

* fix typo

* add test_chunk

* fix bugs

* fix bugs

* polish code

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* rm model zoo

* add Parameter inheritance for ColoParameter

* remove tricks

* remove tricks

* polish

* polish

Co-authored-by: github-actions <github-actions@github.com>

* add load_state_dict for sharded model

* fix bug

* fix bug

* fix ckpt dtype and device

* support load state dict in zero init ctx

* fix bugs

* add Embedding

* Add bert test

* polish

* add check module test

* polish

* polish

* polish

* polish

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [p2p]add object list send recv

* refactor for code reusability

* polish

* add module spec for linear

* polish

* polish

* polish

* [ci] fixed nightly build workflow

* [ci] fixed nightly build workflow

* [ci] fixed nightly build workflow

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [hotfix]fix some bugs caused by size mismatch.

* add warning logs

* polish

* impl ColoDDP for ColoTensor

* polish code

* refactor parallel action

* polish unit tests

Co-authored-by: github-actions <github-actions@github.com>

* polish test_gpt

* update op unit tests

* update test model

* refactor colo-tensor and update linear op

* polish code

* polish code

* update ops and unit tests

* update unit tests

* polish code

* rename dist_spec module

* polish code

* polish code

* remove unneeded import

* fix pipelinable