GitLab

* init a checkpoint dir

* [checkpoint]support resume for cosinewarmuplr

* [checkpoint]add unit test

* fix some bugs but still not OK

* fix bugs

* make it faster

* [checkpoint]support generalized scheduler

* polish

* [tensor] torch function return colotensor

* polish

* fix bugs

* remove debug info

* polish

* polish

* [tensor] test_model pass unittests

* polish

* [hotfix] fx get comm size bug
Co-authored-by: ZhaoYi1222 <zhaoyi9499@gmail.com>

* fix equal assert

* polish

* [test] ignore 8 gpu test

* polish code

* polish workflow

* polish workflow

* reorgnize colotensor directory

* polish code

* add Parameter inheritance for ColoParameter

* remove tricks

* remove tricks

* polish

* polish

* refactor parallel action

* polish unit tests

* polish test_gpt

* update op unit tests

* update test model

* refactor colo-tensor and update linear op

* polish code

* polish code

* update ops and unit tests

* update unit tests

* polish code

* rename dist_spec module

* polish code

* polish code

* remove unneeded import

* fix pipelinable

* derive compute pattern from dist spec

* polish code

* add dist spec

* update linear op

* polish code

* polish code

* update embedding op

* polish unit tests

* polish unit tests

* polish comments

* polish code

* add test_dist_spec_mgr

* polish code

* refactor folder structure

* polish unit tests

* add get_process_group() for TensorSpec

* polish code

* add optimizer to bert test

* polish

* change torch.Parameter to ColoParameter

* fix post assignment for init context

* polish

* polish

* simplify ColoModulize

* simplify ColoModulize

* polish

* polish

* add from_pretrained support and test

* polish

* polish

* polish

* polish

* add 1d vocab loss

* polish

* polish test_model

* polish

* add base bert test

* Add bert test

* polish

* remove test_bert

* polish

* [Tensor] activation is an attr of ColoTensor

* [Tensor] add optimizer

* only detach parameters in context

* polish code

* refine linear and add function to ColoTensor

* add gather for layernorm

* polish

* polish