GitLab

* add linearconv1d test

* add linearconv1d test

* [shardformer] support module saving and loading

* polish code

* support kit use for bert test

* support kit test for gpt2

* [shardformer] adapted T5 and LLaMa test to use kit

* polish code

* add gpt2 test and layer class refactor

* add dropout in gpt2 policy

* fix bert downstream with new api

* remove comment line

* [shardformer] refactored embedding and dropout to parallel module

* polish code

* [shardformer] integrated linear 1D with dtensor

* polish code

* add dist dropout in model

* update docstring and bert policy with dropout

* refactor basepolicy and sharded, update bert

* update format

* update gpt2 policy

* update bert policy

* remove unused code

* update readme for new policy usage

* add downstream model of bert

* remove unused code

test t5

adjust layer attr

* fix bug in slicer, add slicer unit test

* add dropout test

* use pid as dropout seed

* updata dropout test with local pattern

* ad todo

* add bert align test, fix dist loss bug

* forward and backward align

* add ignore index

* add shardformer CI

* add gather_output optional for user in shardconfig

* update readme with optional gather_ouput

* add dist crossentropy loss test, remove unused files

* remove unused file

* remove unused file

* rename the file

* polish code