GitLab

* [test] added spawn decorator

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code

* [zero] refactor low-level zero folder structure

* [zero] fix legacy zero import path

* [zero] fix legacy zero import path

* [zero] remove useless import

* [zero] refactor gemini folder structure

* [zero] refactor gemini folder structure

* [zero] refactor legacy zero import path

* [zero] refactor gemini folder structure

* [zero] refactor gemini folder structure

* [zero] refactor gemini folder structure

* [zero] refactor legacy zero import path

* [zero] fix test import path

* [zero] fix test

* [zero] fix circular import

* [zero] update import

* [DTensor] refactor dtensor with new components

* polish

* [DTensor] refactor LayoutConverter for DTensor

* polish code

* polish docstring

* [hotfix] skip auto checkpointing tests

* fix test name issue

* [autoparallel] refactor sharding spec

* rename function name

* [DTensor] implementation of dtensor

* test layout convert

* polish

* [zero] add strict ddp mode for chunk init

* [gemini] update gpt example

* [zero] add strict ddp mode

* [polish] add comments for strict ddp mode

* [zero] fix test error

* [gemini] polish code

* [testing] remove code

* [gemini] make more robust

* [testing] add beit model

* [beit] fix bugs

* [beit] fix bugs

* [testing] fix bugs

* Add mix-gather

* Add comments

* Add comments

* Polish comments

* Change the global rank assumption

* Add tests

* Add two-step tests

* Fix 10 and 01

* Skip test becasue the number of GPUs

* [autoparallel] shard param and buffer as expected

* fix unit test issue

* [autoparallel] handled illegal sharding strategy

* polish code

* add chunk manager init function

* fix unit tests

* add comment

* add flush=True

* [fix] fixed the collective pattern name for consistency

* polish code

This reverts commit 5be118f4.

* [tensor] use communication autograd func

* change all to all comm spec info

* rename pattern and distinguish fwd/bwd

* polish code

* [tensor] support runtime ShardingSpec apply

* polish code

* polish code

* [tensor] shape consistency output transform path and communication cost

* polish code

* [tensor] added linear implementation for the new sharding spec

* polish code

* [tensor] add shape consistency feature to supportauto sharding spec transform.

* [tensor] remove unused argument in simulator, add doc string for target pair.