Commits · 4fb3c52cf024944b0e443b97b2fa3f3499ca45bb · OpenDAS / ColossalAI

09 Aug, 2022 7 commits
- [zero] add unit test for AgChunk's append, close, access (#1423) · 4fb3c52c
  HELSON authored Aug 09, 2022
  
  4fb3c52c
- [zero] add AgChunk (#1417) · c577ed01
  HELSON authored Aug 09, 2022
  
  c577ed01
- Add FreqAwareEmbeddingBag (#1421) · d209aff6
  Jiarui Fang authored Aug 09, 2022
  
  d209aff6
- [hotfix] zero optim prevents calling inner optim.zero_grad (#1422) · 6df3e19b
  ver217 authored Aug 09, 2022
  
  6df3e19b
- [FAW] add cache manager for the cached embedding (#1419) · 504419d2
  Jiarui Fang authored Aug 09, 2022
  
  504419d2
- [communication] add p2p_v2.py to support communication with List[Any] (#1407) · 44fd3c83
  Kirigaya Kazuto authored Aug 09, 2022
```
* support p2p communication with any type of object | pass test

* reconstruct pipeline schedule with p2p_v2.py(support communication with List[Any]) | pass test

* [communication] add p2p_v2.py to support communication with List[Any]

* Delete _pipeline_schedule_v2.py

* Delete test_cifar_with_data_pipeline_tensor_v2.py

* [engin/schedule] use p2p_v2 to recontruct pipeline_schedule

* [communication] remove print code

* [communication] remove print code
```
  44fd3c83
- Automated submodule synchronization (#1415) · 1590f599
  github-actions[bot] authored Aug 09, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
  1590f599
08 Aug, 2022 2 commits
- Automated submodule synchronization (#1404) · 9b442ecd
  github-actions[bot] authored Aug 08, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
  9b442ecd
- [tensor]build sharding spec to replace distspec in future. (#1405) · 7c96055c
  YuliangLiu0306 authored Aug 08, 2022
  
  7c96055c
05 Aug, 2022 1 commit
- [hotfix] fix CPUAdam kernel nullptr (#1410) · 12b48870
  ver217 authored Aug 05, 2022
  
  12b48870
03 Aug, 2022 1 commit
- Automated submodule synchronization (#1396) · 1e5eb087
  github-actions[bot] authored Aug 03, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
  1e5eb087
02 Aug, 2022 5 commits
- [device] add DeviceMesh class to support logical device layout (#1394) · 0442f940
  YuliangLiu0306 authored Aug 02, 2022
```
* [device] add DeviceMesh class to support logical device layout

* polish code

* add doc string
```
  0442f940
- [zero] ZeroDDP supports controlling outputs' dtype (#1399) · 04c9a86a
  ver217 authored Aug 02, 2022
  
  04c9a86a
- [zero] alleviate memory usage in ZeRODDP state_dict (#1398) · 4e98e938
  HELSON authored Aug 02, 2022
  
  4e98e938
- update nvme on readme (#1397) · 4f5f8f77
  Jiarui Fang authored Aug 02, 2022
  
  4f5f8f77
- [zero] chunk manager allows filtering ex-large params (#1393) · 56b8863b
  ver217 authored Aug 02, 2022
  
  56b8863b
01 Aug, 2022 2 commits
- [fx] fixed torchaudio conformer tracing (#1392) · adf5054f
  Frank Lee authored Aug 01, 2022
  
  adf5054f
- [fx] patched torch.max and data movement operator (#1391) · 7d629392
  Frank Lee authored Aug 01, 2022
```
* [fx] patched torch.max and data movement operator

* polish code
```
  7d629392
30 Jul, 2022 1 commit
- Update README.md · db89600c
  fastalgo authored Jul 30, 2022
  
  db89600c
29 Jul, 2022 6 commits
- [fx] fixed indentation error in checkpointing codegen (#1385) · 89e60d15
  Frank Lee authored Jul 30, 2022
  
  89e60d15
- [hotfix] adapt ProcessGroup and Optimizer to ColoTensor (#1388) · c7221cb2
  HELSON authored Jul 29, 2022
  
  c7221cb2
- [fx] patched torch.full for huggingface opt (#1386) · ad678921
  Frank Lee authored Jul 29, 2022
  
  ad678921
- [hotfix] fix a running error in test_colo_checkpoint.py (#1387) · 527758b2
  HELSON authored Jul 29, 2022
  
  527758b2
- [chunk] add PG check for tensor appending (#1383) · f792507f
  Jiarui Fang authored Jul 29, 2022
  
  f792507f
- [zero] zero optim state_dict takes only_rank_0 (#1384) · 8dced41a
  ver217 authored Jul 29, 2022
```
* zero optim state_dict takes only_rank_0

* fix unit test
```
  8dced41a
28 Jul, 2022 8 commits
- [DDP] test ddp state dict uses more strict threshold (#1382) · 7d5d628e
  ver217 authored Jul 28, 2022
  
  7d5d628e
- [hotfix] fix some bugs during gpt2 testing (#1379) · df544814
  YuliangLiu0306 authored Jul 28, 2022
  
  df544814
- [hotfix] fix zero optim save/load state dict (#1381) · 828b9e5e
  ver217 authored Jul 28, 2022
  
  828b9e5e
- [checkpoint] add kwargs for load_state_dict (#1374) · b6fd165f
  HELSON authored Jul 28, 2022
  
  b6fd165f
- Automated submodule synchronization (#1380) · 50dec605
  github-actions[bot] authored Jul 28, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
  50dec605
- [hotfix] fix zero ddp buffer cast (#1376) · 83328329
  ver217 authored Jul 28, 2022
```
* fix zero ddp buffer cast

* fix zero ddp ignore params
```
  83328329
- fix zero ddp state dict (#1378) · 5d5031e9
  ver217 authored Jul 28, 2022
  
  5d5031e9
- [util] standard checkpoint function naming (#1377) · 0c1a16ea
  Frank Lee authored Jul 28, 2022
  
  0c1a16ea
27 Jul, 2022 3 commits

[fx] update split module pass and add customized policy (#1373) · 52bc2dc2

YuliangLiu0306 authored Jul 27, 2022

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx]update split module pass and add customized policy

52bc2dc2

[fx] add torchaudio test (#1369) · be229217

Super Daniel authored Jul 27, 2022

* [fx]add torchaudio test

* [fx]add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test and test patches

* Delete ~

* [fx] add patches and patches test

* [fx] add patches and patches test

* [fx] fix patches

* [fx] fix rnn patches

* [fx] fix rnn patches

* [fx] fix rnn patches

* [fx] fix rnn patches

* [fx] merge upstream

* [fx] fix import errors

be229217

Automated submodule synchronization (#1372) · fb6f0859
github-actions[bot] authored Jul 27, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
fb6f0859

26 Jul, 2022 4 commits

[fx] Add colotracer compatibility test on torchrec (#1370) · bb640ec7
Boyuan Yao authored Jul 26, 2022

bb640ec7

[nvme] CPUAdam and HybridAdam support NVMe offload (#1360) · c415240d

ver217 authored Jul 26, 2022

* impl nvme optimizer

* update cpu adam

* add unit test

* update hybrid adam

* update docstr

* add TODOs

* update CI

* fix CI

* fix CI

* fix CI path

* fix CI path

* fix CI path

* fix install tensornvme

* fix CI

* fix CI path

* fix CI env variables

* test CI

* test CI

* fix CI

* fix nvme optim __del__

* fix adam __del__

* fix nvme optim

* fix CI env variables

* fix nvme optim import

* test CI

* test CI

* fix CI

c415240d

[checkpoint] use args, kwargs in save_checkpoint, load_checkpoint (#1368) · 84632906
HELSON authored Jul 26, 2022

84632906
Automated submodule synchronization (#1364) · c491c2a9
github-actions[bot] authored Jul 26, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
c491c2a9