GitLab

Co-authored-by: 何晓昕 <cautious@r-205-106-25-172.comp.nus.edu.sg>

* place params on cpu after zero init context

* polish code

* fix merge conflict

modify unit test

remove unnessesary log info

reformat file

* remove unused module

* remove unnecessary sync function

* change doc string style from Google to Sphinx

* add zero init context

* add more flags for zero init context
fix bug of repeated converting param to ShardedParamV2

* polish code

Fixed bug for learning rate scheduler