GitLab

* Refactor MoE Manager setup method

* unshard optim ckpt

* optim io

* update transformer version

* update requirements

* update ckpt

* update ckpt

* update ckpt

* fix engine

* fix engine

* update moe module
* support openmoe

* [misc] update pre-commit

* [misc] run pre-commit

* [misc] remove useless configuration files

* [misc] ignore cuda for clang-format

* [test] added spawn decorator

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code

* [test] refactored with the new rerun decorator

* polish test case