GitLab

* [fx] added torchvision model tracing testing

* remove unused imports

* [fx] added testing for all gpt variants

* polish code

* polish code

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx]add uniform policy

* [fx] added testing for all bert variants

* polish code

* [fx] supported model tracing for huggingface bert

* polish test

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [context]support arbitary module materialization.

* [test]add numerical check for lazy init context.

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [context]use meta tensor to init model lazily.

* polish

* make module with device kwargs bypass the normal init.

* change unit test to adapt updated context.

* [fx] supported data-dependent control flow in model tracing

* polish code

* add reducer

* update colo ddp with reducer

* polish unit test

* polish unit test

* sharded optim supports loading local state dict

* polish code

* add unit test

* zero optim supports loading local state dict

* polish code

* add unit test

* add more element-wise ops

* update test_op

* polish unit test

* dist spec s2s uses all-to-all

* update unit test

* add sanity check

* polish unitest test with titans

* add sanity check for DistMgr

* add sanity check
Co-authored-by: jiaruifang <fangjiarui123@gmail.com>

* ColoDDP supports overwriting default process group

* rename ColoDDPV2 to ZeroDDP

* add docstr for ZeroDDP

* polish docstr

* add save/load state dict for ColoDDP

* add unit test

* refactor unit test folder

* polish unit test

* rename unit test

* fix param op hook

* update zero tp test

* fix bugs

* add set_params_to_ignore for ColoDDP

* polish code

* fix zero hook v2

* add unit test

* polish docstr

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* feature/add autoparallel passes