GitLab

Co-authored-by: github-actions <github-actions@github.com>

* change torch.Parameter to ColoParameter

* fix post assignment for init context

* polish

* polish

* simplify ColoModulize

* simplify ColoModulize

* polish

* polish

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [pipelinable]use pipelinable to support GPT model.

* fix a bug caused by ShardedModel

* polish

* fix front func list

Co-authored-by: github-actions <github-actions@github.com>

* colo tensor overrides mul

* polish code

* hijack addmm for colo tensor

* fix bugs

* polish unit test

* polish comments

* add from_pretrained support and test

* polish

* polish

* polish

* polish

* support more cuda archs

* polish code

* add 1d vocab loss

* polish

* polish test_model

* polish

* add base bert test

* Add bert test

* polish

* remove test_bert

* polish

Co-authored-by: github-actions <github-actions@github.com>

* [ci] added wheel build scripts

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* [ci] polish wheel build scripts

Co-authored-by: github-actions <github-actions@github.com>

* [Tensor] activation is an attr of ColoTensor

* [Tensor] add optimizer

* only detach parameters in context

* polish code

* refine linear and add function to ColoTensor

* add gather for layernorm

* polish

* polish

* [Tensor] make ColoTensor more robust for getattr

* polish

* polish

* add lru cache

* polish code

* update unit test

* fix sharded optim