GitLab

* add auto-offload feature

* polish code

* fix syn offload runtime pass bug

* add offload example

* fix offload testing bug

* fix example testing bug

* add alpa dp split

* add alpa dp split

* use fwd+bwd instead of fwd only

---------
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>

Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>

* [autoparallel] integrate device mesh initialization into autoparallelize

* add megatron solution

* update gpt autoparallel examples with latest api

* adapt beta value to fit the current computation cost