Commits · c415240db6a3d286e707edec31a770a166ddc2fc · OpenDAS / ColossalAI

26 Jul, 2022 6 commits

[nvme] CPUAdam and HybridAdam support NVMe offload (#1360) · c415240d

ver217 authored Jul 26, 2022

* impl nvme optimizer

* update cpu adam

* add unit test

* update hybrid adam

* update docstr

* add TODOs

* update CI

* fix CI

* fix CI

* fix CI path

* fix CI path

* fix CI path

* fix install tensornvme

* fix CI

* fix CI path

* fix CI env variables

* test CI

* test CI

* fix CI

* fix nvme optim __del__

* fix adam __del__

* fix nvme optim

* fix CI env variables

* fix nvme optim import

* test CI

* test CI

* fix CI

c415240d

[checkpoint] use args, kwargs in save_checkpoint, load_checkpoint (#1368) · 84632906
HELSON authored Jul 26, 2022

84632906
Automated submodule synchronization (#1364) · c491c2a9
github-actions[bot] authored Jul 26, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
c491c2a9

[fx]add gpt2 passes for pipeline performance test (#1366) · 55428166

YuliangLiu0306 authored Jul 26, 2022

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx]add gpt2 passes for pipeline performance test

55428166

[colotensor] use cpu memory to store state_dict (#1367) · 87775a06
HELSON authored Jul 26, 2022

87775a06
[hotfix] fix no optimizer in save/load (#1363) · 943a9632
HELSON authored Jul 26, 2022

943a9632

25 Jul, 2022 4 commits
- [fx] added activation checkpoint codegen support for torch < 1.12 (#1359) · cd063ac3
  Frank Lee authored Jul 25, 2022
  
  cd063ac3
- [unit test] add megatron init test in zero_optim (#1358) · 44178041
  HELSON authored Jul 25, 2022
  
  44178041
- [hotfix] fix megatron_init in test_gpt2.py (#1357) · 7a065dc9
  HELSON authored Jul 25, 2022
  
  7a065dc9
- [fx] added activation checkpoint codegen (#1355) · 644582ee
  Frank Lee authored Jul 25, 2022
  
  644582ee
21 Jul, 2022 9 commits
- [docker] add tensornvme in docker (#1354) · 38fd8844
  ver217 authored Jul 21, 2022
```
* add tensornvme in docker

* fix dockerfile

* fix dockerfile
```
  38fd8844
- fix zero optim backward_by_grad and save/load (#1353) · 6b43c789
  ver217 authored Jul 21, 2022
  
  6b43c789
- [doc] update rst and docstring (#1351) · d068af81
  ver217 authored Jul 21, 2022
```
* update rst

* add zero docstr

* fix docstr

* remove fx.tracer.meta_patch

* fix docstr

* fix docstr

* update fx rst

* fix fx docstr

* remove useless rst
```
  d068af81
- [fx] fixed apex normalization patch exception (#1352) · 274c1a3b
  Frank Lee authored Jul 21, 2022
  
  274c1a3b
- [checkpoint] sharded optim save/load grad scaler (#1350) · ce470ba3
  ver217 authored Jul 21, 2022
  
  ce470ba3
- [fx] added activation checkpointing annotation (#1349) · 05fae1fd
  Frank Lee authored Jul 21, 2022
```
* [fx] added activation checkpointing annotation

* polish code

* polish code
```
  05fae1fd
- [fx] update MetaInforProp pass to process more complex node.meta (#1344) · 051592c6
  YuliangLiu0306 authored Jul 21, 2022
```
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx] update MetaInforProp pass to process more complex node.meta
```
  051592c6
- [colotensor] add Tensor.view op and its unit test (#1343) · 7a8702c0
  HELSON authored Jul 21, 2022
```
[colotensor] add megatron initialization for gpt2
```
  7a8702c0
- Automated submodule synchronization (#1348) · 6160a1d6
  github-actions[bot] authored Jul 21, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
  6160a1d6
20 Jul, 2022 3 commits

[NFC] add OPT (#1345) · 92b0b139
binmakeswell authored Jul 20, 2022

92b0b139

[fx] refactor tracer to trace complete graph (#1342) · 942c8cd1

YuliangLiu0306 authored Jul 20, 2022

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx] refactor tracer to trace complete graph

* add comments and solve conflicts.

942c8cd1

[fx] tested the complete workflow for auto-parallel (#1336) · 2cc1175c
Frank Lee authored Jul 20, 2022
```
* [fx] tested the complete workflow for auto-parallel

* polish code

* polish code

* polish code
```
2cc1175c

19 Jul, 2022 4 commits
- [fx]refactor tracer (#1335) · 4631fef8
  YuliangLiu0306 authored Jul 19, 2022
  
  4631fef8
- [refactor] refactor ColoTensor's unit tests (#1340) · bf5066fb
  HELSON authored Jul 19, 2022
  
  bf5066fb
- [checkpoint] use gather_tensor in checkpoint and update its unit test (#1339) · f92c100d
  HELSON authored Jul 19, 2022
  
  f92c100d
- [fx] recovered skipped pipeline tests (#1338) · f3ce7b83
  Frank Lee authored Jul 19, 2022
  
  f3ce7b83
18 Jul, 2022 4 commits
- [hotfix] ZeroDDP use new process group (#1333) · 0c51ff2c
  ver217 authored Jul 18, 2022
```
* process group supports getting ranks in group

* chunk mgr receives a process group

* update unit test

* fix unit tests
```
  0c51ff2c
- [workflow] update docker build workflow to use proxy (#1334) · 11d1436a
  Frank Lee authored Jul 18, 2022
  
  11d1436a
- [fx] fixed compatiblity issue with torch 1.10 (#1331) · 75abc75c
  Frank Lee authored Jul 18, 2022
  
  75abc75c
- [workflow] update 8-gpu test to use torch 1.11 (#1332) · 069d6fdc
  Frank Lee authored Jul 18, 2022
  
  069d6fdc
17 Jul, 2022 1 commit
- Update README.md · 7857fd76
  fastalgo authored Jul 16, 2022
  
  7857fd76
15 Jul, 2022 9 commits
- [test] removed outdated unit test for meta context (#1329) · 169954f8
  Frank Lee authored Jul 15, 2022
  
  169954f8
- [hotfix] shared model returns cpu state_dict (#1328) · 7a053671
  ver217 authored Jul 15, 2022
  
  7a053671
- [fx] fixed unit tests for torch 1.12 (#1327) · b2475d8c
  Frank Lee authored Jul 15, 2022
  
  b2475d8c
- [hotfix] fix ddp for unit test test_gpt2 (#1326) · d49708ae
  HELSON authored Jul 15, 2022
  
  d49708ae
- [utils] integrated colotensor with lazy init context (#1324) · 250be4d3
  Frank Lee authored Jul 15, 2022
```
* [utils] integrated colotensor with lazy init context

* polish code

* polish code

* polish code
```
  250be4d3
- [workflow] roll back to use torch 1.11 for unit testing (#1325) · 659a7407
  Frank Lee authored Jul 15, 2022
  
  659a7407
- [workflow] fixed trigger condition for 8-gpu unit test (#1323) · 4d5dbf48
  Frank Lee authored Jul 15, 2022
  
  4d5dbf48
- [fx] add balanced policy v2 (#1251) · e8acf55e
  YuliangLiu0306 authored Jul 15, 2022
```
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx] add balanced policy v2

* add unittest
```
  e8acf55e
- [fx] Add unit test and fix bugs for transform_mlp_pass (#1299) · ca2d3f28
  XYE authored Jul 15, 2022
```
* add test and fix bugs

* add functions back

* add comments
```
  ca2d3f28