Commits · ad678921db5fc9c8080885b563974f604900e4e1 · OpenDAS / ColossalAI

29 Jul, 2022 4 commits
- [fx] patched torch.full for huggingface opt (#1386) · ad678921
  Frank Lee authored Jul 29, 2022
  
  ad678921
- [hotfix] fix a running error in test_colo_checkpoint.py (#1387) · 527758b2
  HELSON authored Jul 29, 2022
  
  527758b2
- [chunk] add PG check for tensor appending (#1383) · f792507f
  Jiarui Fang authored Jul 29, 2022
  
  f792507f
- [zero] zero optim state_dict takes only_rank_0 (#1384) · 8dced41a
  ver217 authored Jul 29, 2022
```
* zero optim state_dict takes only_rank_0

* fix unit test
```
  8dced41a
28 Jul, 2022 8 commits
- [DDP] test ddp state dict uses more strict threshold (#1382) · 7d5d628e
  ver217 authored Jul 28, 2022
  
  7d5d628e
- [hotfix] fix some bugs during gpt2 testing (#1379) · df544814
  YuliangLiu0306 authored Jul 28, 2022
  
  df544814
- [hotfix] fix zero optim save/load state dict (#1381) · 828b9e5e
  ver217 authored Jul 28, 2022
  
  828b9e5e
- [checkpoint] add kwargs for load_state_dict (#1374) · b6fd165f
  HELSON authored Jul 28, 2022
  
  b6fd165f
- Automated submodule synchronization (#1380) · 50dec605
  github-actions[bot] authored Jul 28, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
  50dec605
- [hotfix] fix zero ddp buffer cast (#1376) · 83328329
  ver217 authored Jul 28, 2022
```
* fix zero ddp buffer cast

* fix zero ddp ignore params
```
  83328329
- fix zero ddp state dict (#1378) · 5d5031e9
  ver217 authored Jul 28, 2022
  
  5d5031e9
- [util] standard checkpoint function naming (#1377) · 0c1a16ea
  Frank Lee authored Jul 28, 2022
  
  0c1a16ea
27 Jul, 2022 3 commits

[fx] update split module pass and add customized policy (#1373) · 52bc2dc2

YuliangLiu0306 authored Jul 27, 2022

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx]update split module pass and add customized policy

52bc2dc2

[fx] add torchaudio test (#1369) · be229217

Super Daniel authored Jul 27, 2022

* [fx]add torchaudio test

* [fx]add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test and test patches

* Delete ~

* [fx] add patches and patches test

* [fx] add patches and patches test

* [fx] fix patches

* [fx] fix rnn patches

* [fx] fix rnn patches

* [fx] fix rnn patches

* [fx] fix rnn patches

* [fx] merge upstream

* [fx] fix import errors

be229217

Automated submodule synchronization (#1372) · fb6f0859
github-actions[bot] authored Jul 27, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
fb6f0859

26 Jul, 2022 7 commits
- [fx] Add colotracer compatibility test on torchrec (#1370) · bb640ec7
  Boyuan Yao authored Jul 26, 2022
  
  bb640ec7
- [nvme] CPUAdam and HybridAdam support NVMe offload (#1360) · c415240d
  ver217 authored Jul 26, 2022
```
* impl nvme optimizer

* update cpu adam

* add unit test

* update hybrid adam

* update docstr

* add TODOs

* update CI

* fix CI

* fix CI

* fix CI path

* fix CI path

* fix CI path

* fix install tensornvme

* fix CI

* fix CI path

* fix CI env variables

* test CI

* test CI

* fix CI

* fix nvme optim __del__

* fix adam __del__

* fix nvme optim

* fix CI env variables

* fix nvme optim import

* test CI

* test CI

* fix CI
```
  c415240d
- [checkpoint] use args, kwargs in save_checkpoint, load_checkpoint (#1368) · 84632906
  HELSON authored Jul 26, 2022
  
  84632906
- Automated submodule synchronization (#1364) · c491c2a9
  github-actions[bot] authored Jul 26, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
  c491c2a9
- [fx]add gpt2 passes for pipeline performance test (#1366) · 55428166
  YuliangLiu0306 authored Jul 26, 2022
```
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx]add gpt2 passes for pipeline performance test
```
  55428166
- [colotensor] use cpu memory to store state_dict (#1367) · 87775a06
  HELSON authored Jul 26, 2022
  
  87775a06
- [hotfix] fix no optimizer in save/load (#1363) · 943a9632
  HELSON authored Jul 26, 2022
  
  943a9632
25 Jul, 2022 4 commits
- [fx] added activation checkpoint codegen support for torch < 1.12 (#1359) · cd063ac3
  Frank Lee authored Jul 25, 2022
  
  cd063ac3
- [unit test] add megatron init test in zero_optim (#1358) · 44178041
  HELSON authored Jul 25, 2022
  
  44178041
- [hotfix] fix megatron_init in test_gpt2.py (#1357) · 7a065dc9
  HELSON authored Jul 25, 2022
  
  7a065dc9
- [fx] added activation checkpoint codegen (#1355) · 644582ee
  Frank Lee authored Jul 25, 2022
  
  644582ee
21 Jul, 2022 9 commits
- [docker] add tensornvme in docker (#1354) · 38fd8844
  ver217 authored Jul 21, 2022
```
* add tensornvme in docker

* fix dockerfile

* fix dockerfile
```
  38fd8844
- fix zero optim backward_by_grad and save/load (#1353) · 6b43c789
  ver217 authored Jul 21, 2022
  
  6b43c789
- [doc] update rst and docstring (#1351) · d068af81
  ver217 authored Jul 21, 2022
```
* update rst

* add zero docstr

* fix docstr

* remove fx.tracer.meta_patch

* fix docstr

* fix docstr

* update fx rst

* fix fx docstr

* remove useless rst
```
  d068af81
- [fx] fixed apex normalization patch exception (#1352) · 274c1a3b
  Frank Lee authored Jul 21, 2022
  
  274c1a3b
- [checkpoint] sharded optim save/load grad scaler (#1350) · ce470ba3
  ver217 authored Jul 21, 2022
  
  ce470ba3
- [fx] added activation checkpointing annotation (#1349) · 05fae1fd
  Frank Lee authored Jul 21, 2022
```
* [fx] added activation checkpointing annotation

* polish code

* polish code
```
  05fae1fd
- [fx] update MetaInforProp pass to process more complex node.meta (#1344) · 051592c6
  YuliangLiu0306 authored Jul 21, 2022
```
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx] update MetaInforProp pass to process more complex node.meta
```
  051592c6
- [colotensor] add Tensor.view op and its unit test (#1343) · 7a8702c0
  HELSON authored Jul 21, 2022
```
[colotensor] add megatron initialization for gpt2
```
  7a8702c0
- Automated submodule synchronization (#1348) · 6160a1d6
  github-actions[bot] authored Jul 21, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
  6160a1d6
20 Jul, 2022 3 commits

[NFC] add OPT (#1345) · 92b0b139
binmakeswell authored Jul 20, 2022

92b0b139

[fx] refactor tracer to trace complete graph (#1342) · 942c8cd1

YuliangLiu0306 authored Jul 20, 2022

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx] refactor tracer to trace complete graph

* add comments and solve conflicts.

942c8cd1

[fx] tested the complete workflow for auto-parallel (#1336) · 2cc1175c
Frank Lee authored Jul 20, 2022
```
* [fx] tested the complete workflow for auto-parallel

* polish code

* polish code

* polish code
```
2cc1175c

19 Jul, 2022 2 commits
- [fx]refactor tracer (#1335) · 4631fef8
  YuliangLiu0306 authored Jul 19, 2022
  
  4631fef8
- [refactor] refactor ColoTensor's unit tests (#1340) · bf5066fb
  HELSON authored Jul 19, 2022
  
  bf5066fb