GitLab

* sharded optim supports loading local state dict

* polish code

* add unit test

* zero optim supports loading local state dict

* polish code

* add unit test

* add more element-wise ops

* update test_op

* polish unit test

* dist spec s2s uses all-to-all

* update unit test

* add sanity check

* polish unitest test with titans

* add sanity check for DistMgr

* add sanity check
Co-authored-by: jiaruifang <fangjiarui123@gmail.com>

* ColoDDP supports overwriting default process group

* rename ColoDDPV2 to ZeroDDP

* add docstr for ZeroDDP

* polish docstr

* add save/load state dict for ColoDDP

* add unit test

* refactor unit test folder

* polish unit test

* rename unit test

* fix param op hook

* update zero tp test

* fix bugs

* add set_params_to_ignore for ColoDDP

* polish code

* fix zero hook v2

* add unit test

* polish docstr

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* feature/add autoparallel passes

* [fx] added unit test for coloproxy

* polish code

* polish code

* update gemini mgr

* update chunk

* add docstr

* polish placement policy

* update test chunk

* update test zero

* polish unit test

* remove useless unit test

* add placement policy

* add gemini mgr

* update mem stats collector

* update zero

* update zero optim

* fix bugs

* zero optim monitor os

* polish unit test

* polish unit test

* add assert

* [pipeline] refactor the pipeline module

* polish code

* [context] support lazy init of module

* polish code

* polish chunk manager

* polish unit test

* impl add_extern_static_tensor for chunk mgr

* add mem stats collector v2

* polish code

* polish unit test

* polish code

* polish get chunks

* fix equal assert

* polish

* [test] skip tests when not enough GPUs are detected

* polish code

* polish code

* [test] ignore 8 gpu test

* polish code

* polish workflow

* polish workflow

* Add CPU 1d row embedding

* polish

* reorgnize colotensor directory

* polish code

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* refactor ppschedule to support tensor list

* polish

* add zero optimizer

* torch ok

* unit test ok

* polish code

* fix bugs

* polish unit test

* polish zero optim

* polish colo ddp v2

* refactor folder structure

* add comment

* polish unit test

* polish zero optim

* polish unit test

* impl chunk manager

* impl param op hook

* add reduce_chunk

* add zero hook v2

* add zero dp

* fix TensorInfo

* impl load balancing when using zero without chunk

* fix zero hook

* polish chunk

* fix bugs

* ddp ok

* zero ok

* polish code

* fix bugs about load balancing

* polish code

* polish code

* add ene-to-end test

* polish code

* polish code

* polish code

* fix typo

* add test_chunk

* fix bugs

* fix bugs

* polish code

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* rm model zoo

* add Parameter inheritance for ColoParameter

* remove tricks

* remove tricks

* polish

* polish

* add Embedding

* Add bert test

* polish

* add check module test

* polish

* polish

* polish

* polish