Commits · ac88de6dfc69bc59d4cadbd6432b0b818ca37e60 · OpenDAS / ColossalAI

22 Apr, 2022 3 commits
- [WIP] Applying ColoTensor on TP-1D-row Linear. (#831) · ac88de6d
  Jiarui Fang authored Apr 22, 2022
```
* revert zero tensors back

* [tensor] init row 1d linear
```
  ac88de6d
- revert zero tensors back (#829) · 595bedf7
  Jiarui Fang authored Apr 22, 2022
  
  595bedf7
- [tensor] ZeRO use ColoTensor as the base class. (#828) · 294a6060
  Jiarui Fang authored Apr 22, 2022
```
* [refactor] moving InsertPostInitMethodToModuleSubClasses to utils.

* [tensor] ZeRO use ColoTensor as the base class.

* polish
```
  294a6060
21 Apr, 2022 9 commits

[tensor]fix test_linear (#826) · 8e6fdb4f
Ziyue Jiang authored Apr 21, 2022

8e6fdb4f
[tensor] fix kwargs in colo_tensor torch_funtion (#825) · 1a9e2c2d
Ziyue Jiang authored Apr 21, 2022

1a9e2c2d
[refactor] moving InsertPostInitMethodToModuleSubClasses to utils. (#824) · eb1b8990
Jiarui Fang authored Apr 21, 2022

eb1b8990
[tensor] lazy init (#823) · 2ecc3d7a
Jiarui Fang authored Apr 21, 2022

2ecc3d7a

[Tensor] update ColoTensor torch_function (#822) · 68dcd51d

Jiarui Fang authored Apr 21, 2022

* Revert "[zero] add ZeroTensorShardStrategy (#793)"

This reverts commit 88759e28.

* [gemini] set cpu memory capacity

* [log] local throughput collecting

* polish

* polish

* polish

* polish code

* polish

* polish code

* add a new tensor structure and override linear for it

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* [tensor] renaming and reorganize directory structure.

* rm useless dir

* polish

* polish

* [tensor] hander the function not wrapped

* polish

68dcd51d

[Tensor] apply ColoTensor on Torch functions (#821) · 660d2d1f

Jiarui Fang authored Apr 21, 2022

* Revert "[zero] add ZeroTensorShardStrategy (#793)"

This reverts commit 88759e28.

* [gemini] set cpu memory capacity

* [log] local throughput collecting

* polish

* polish

* polish

* polish code

* polish

* polish code

* add a new tensor structure and override linear for it

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* [tensor] renaming and reorganize directory structure.

* rm useless dir

* polish

* polish

* [tensor] hander the function not wrapped

660d2d1f

[tensor] reorganize files (#820) · 0ce8924c
Jiarui Fang authored Apr 21, 2022

0ce8924c

[gemini] a new tensor structure (#818) · ab962b97

Jiarui Fang authored Apr 21, 2022

* Revert "[zero] add ZeroTensorShardStrategy (#793)"

This reverts commit 88759e28.

* [gemini] set cpu memory capacity

* [log] local throughput collecting

* polish

* polish

* polish

* polish code

* polish

* polish code

* add a new tensor structure and override linear for it

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

ab962b97

Automated submodule synchronization (#819) · 413ce30c
github-actions[bot] authored Apr 21, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
413ce30c

20 Apr, 2022 7 commits
- Automated submodule synchronization (#810) · 9aae4197
  github-actions[bot] authored Apr 20, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
  9aae4197
- Merge pull request #815 from FrankLeeeee/feature/check-cli · e1b38998
  YuliangLiu0306 authored Apr 20, 2022
```
[cli] added check installation cli
```
  e1b38998
- [cli] added check installation cli · 70ed11d0
  FrankLeeeee authored Apr 20, 2022
  
  70ed11d0
- Merge pull request #812 from FrankLeeeee/feature/cli · c7eca40f
  YuliangLiu0306 authored Apr 20, 2022
```
[cli] fixed single-node process launching
```
  c7eca40f
- [gemini] collect cpu-gpu moving volume in each iteration (#813) · 3ddbd1bc
  Jiarui Fang authored Apr 20, 2022
  
  3ddbd1bc
- [cli] fixed single-node process launching · d522cb70
  FrankLeeeee authored Apr 20, 2022
  
  d522cb70
- [log] local throughput metrics (#811) · 61c20b44
  Jiarui Fang authored Apr 20, 2022
```
* Revert "[zero] add ZeroTensorShardStrategy (#793)"

This reverts commit 88759e28.

* [gemini] set cpu memory capacity

* [log] local throughput collecting

* polish

* polish

* polish

* polish code

* polish
```
  61c20b44
19 Apr, 2022 12 commits
- [DO NOT MERGE] [zero] init fp16 params directly in ZeroInitContext (#808) · dd92b90a
  ver217 authored Apr 19, 2022
```
* init fp16 param directly

* polish code
```
  dd92b90a
- [gemini] APIs to set cpu memory capacity (#809) · 227d1cd4
  Jiarui Fang authored Apr 19, 2022
  
  227d1cd4
- Merge pull request #807 from FrankLeeeee/feature/cli · f6dcd23f
  YuliangLiu0306 authored Apr 19, 2022
```
[cli] fixed a bug in user args and refactored the module structure
```
  f6dcd23f
- [cli] fixed a bug in user args and refactored the module structure · f63e91d2
  FrankLeeeee authored Apr 19, 2022
  
  f63e91d2
- Revert "[zero] add ZeroTensorShardStrategy (#793)" (#806) · e761ad2c
  Jiarui Fang authored Apr 19, 2022
  
  e761ad2c
- [zero] add ZeroTensorShardStrategy (#793) · 88759e28
  HELSON authored Apr 19, 2022
  
  88759e28
- [refactor] moving grad acc logic to engine (#804) · 681addb5
  Jiarui Fang authored Apr 19, 2022
  
  681addb5
- [cli] add missing requirement (#805) · 05d9ae59
  Frank Lee authored Apr 19, 2022
  
  05d9ae59
- [cli] added micro benchmarking for tp (#789) · de2f581d
  YuliangLiu0306 authored Apr 19, 2022
```
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [CLI]add cli benchmark feature

* fix CodeFactor issues.

* refactor the module structure.
```
  de2f581d
- [cli] added distributed launcher command (#791) · cfadc9df
  YuliangLiu0306 authored Apr 19, 2022
```
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [CLI]add cli launcher feature

* remove testing message used during developing

* refactor the module structure.
```
  cfadc9df
- [log] display tflops if available (#802) · 97cd9b03
  Jiarui Fang authored Apr 19, 2022
  
  97cd9b03
- [refactor] moving memtracer to gemini (#801) · 4d9332b4
  Jiarui Fang authored Apr 19, 2022
  
  4d9332b4
18 Apr, 2022 3 commits
- [hotfix] fix grad offload when enabling reuse_fp16_shard · 8711c706
  Jiarui Fang authored Apr 18, 2022
  
  8711c706
- fix grad offload when enabling reuse_fp16_shard · f1fa1a67
  ver217 authored Apr 18, 2022
  
  f1fa1a67
- [hotfix] fix memory leak in zero (#781) · 4c4388c4
  HELSON authored Apr 18, 2022
  
  4c4388c4
16 Apr, 2022 3 commits
- [TP] change the check assert in split batch 2d (#772) · 4b01da24
  Ziyue Jiang authored Apr 16, 2022
  
  4b01da24
- [gemini] fix auto tensor placement policy (#775) · 846406a0
  ver217 authored Apr 16, 2022
  
  846406a0
- update version (#779) · 38102cf6
  ver217 authored Apr 16, 2022
  
  38102cf6
15 Apr, 2022 1 commit
- [zero] refactor shard and gather operation (#773) · a65cbb7e
  HELSON authored Apr 15, 2022
  
  a65cbb7e
14 Apr, 2022 2 commits
- [test] refactored with the new rerun decorator (#763) · 5a1a095b
  Frank Lee authored Apr 15, 2022
```
* [test] refactored with the new rerun decorator

* polish test case
```
  5a1a095b
- [readme] sync CN readme (#766) · deaf99f4
  binmakeswell authored Apr 14, 2022
  
  deaf99f4