GitLab

* [zero] refactor low-level zero folder structure

* [zero] fix legacy zero import path

* [zero] fix legacy zero import path

* [zero] remove useless import

* [zero] refactor gemini folder structure

* [zero] refactor gemini folder structure

* [zero] refactor legacy zero import path

* [zero] refactor gemini folder structure

* [zero] refactor gemini folder structure

* [zero] refactor gemini folder structure

* [zero] refactor legacy zero import path

* [zero] fix test import path

* [zero] fix test

* [zero] fix circular import

* [zero] update import

* [test] fixed gemini plugin test

* polish code

* polish code

* [zero] add zero wrappers

* change names

* add wrapper functions to init

* [zero] fix gradient clipping in hybrid parallelism

* [testing] change model name to avoid pytest warning

* [hotfix] fix unit testing

* [testing] add beit model

* [beit] fix bugs

* [beit] fix bugs

* [testing] fix bugs

* [zero] fix memory leak for zero2

* [zero] test gradient accumulation

* [zero] remove grad clip test

* add zero1&2 optimizer

* rename test ditectory

* rename test files

* change tolerance in test

This reverts commit 5be118f4.

* remove forced FP32 modules

* correct no_shard-contexts' positions

* zero optim state_dict takes only_rank_0

* fix unit test

[colotensor] add megatron initialization for gpt2

* process group supports getting ranks in group

* chunk mgr receives a process group

* update unit test

* fix unit tests

* sharded optim supports loading local state dict

* polish code

* add unit test

* zero optim supports loading local state dict

* polish code

* add unit test

* [test] ignore 8 gpu test

* polish code

* polish workflow

* polish workflow

* refactor StatefulTensor, tensor utilities

* add unitest for GeminiMemoryManager

* [test] refactored with the new rerun decorator

* polish test case