Commits · cfadc9df8e7bf57f052793389dc5a96f625e370b · OpenDAS / ColossalAI

19 Apr, 2022 3 commits
- [cli] added distributed launcher command (#791) · cfadc9df
  YuliangLiu0306 authored Apr 19, 2022
```
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [CLI]add cli launcher feature

* remove testing message used during developing

* refactor the module structure.
```
  cfadc9df
- [log] display tflops if available (#802) · 97cd9b03
  Jiarui Fang authored Apr 19, 2022
  
  97cd9b03
- [refactor] moving memtracer to gemini (#801) · 4d9332b4
  Jiarui Fang authored Apr 19, 2022
  
  4d9332b4
18 Apr, 2022 3 commits
- [hotfix] fix grad offload when enabling reuse_fp16_shard · 8711c706
  Jiarui Fang authored Apr 18, 2022
  
  8711c706
- fix grad offload when enabling reuse_fp16_shard · f1fa1a67
  ver217 authored Apr 18, 2022
  
  f1fa1a67
- [hotfix] fix memory leak in zero (#781) · 4c4388c4
  HELSON authored Apr 18, 2022
  
  4c4388c4
16 Apr, 2022 3 commits
- [TP] change the check assert in split batch 2d (#772) · 4b01da24
  Ziyue Jiang authored Apr 16, 2022
  
  4b01da24
- [gemini] fix auto tensor placement policy (#775) · 846406a0
  ver217 authored Apr 16, 2022
  
  846406a0
- update version (#779) · 38102cf6
  ver217 authored Apr 16, 2022
  
  38102cf6
15 Apr, 2022 1 commit
- [zero] refactor shard and gather operation (#773) · a65cbb7e
  HELSON authored Apr 15, 2022
  
  a65cbb7e
14 Apr, 2022 15 commits
- [test] refactored with the new rerun decorator (#763) · 5a1a095b
  Frank Lee authored Apr 15, 2022
```
* [test] refactored with the new rerun decorator

* polish test case
```
  5a1a095b
- [readme] sync CN readme (#766) · deaf99f4
  binmakeswell authored Apr 14, 2022
  
  deaf99f4
- polish sharded optim docstr and warning (#770) · 6e553748
  ver217 authored Apr 14, 2022
  
  6e553748
- fix the ckpt bugs when using DDP (#769) · 80e37eec
  LuGY authored Apr 14, 2022
  
  80e37eec
- [readme] polish readme (#764) · 1f698f44
  Jiarui Fang authored Apr 14, 2022
```
* [readme] polish readme

* centering image
```
  1f698f44
- [compatibility] used backward-compatible API for global process group (#758) · 920fe315
  Frank Lee authored Apr 14, 2022
  
  920fe315
- [test] added a decorator for address already in use error with backward compatibility (#760) · 4ea49cb5
  Frank Lee authored Apr 14, 2022
```
* [test] added a decorator for address already in use error with backward compatibility

* [test] added a decorator for address already in use error with backward compatibility
```
  4ea49cb5
- [gemini] init genimi individual directory (#754) · 10ef8afd
  Jiarui Fang authored Apr 14, 2022
  
  10ef8afd
- [hotfix] fix test_stateful_tensor_mgr (#762) · dcca614e
  ver217 authored Apr 14, 2022
  
  dcca614e
- Automated submodule synchronization (#751) · 6978980f
  github-actions[bot] authored Apr 14, 2022
```
Co-authored-by: github-actions <github-actions@github.com>
```
  6978980f
- [hotfix] fix reuse_fp16_shard of sharded model (#756) · a93a7d73
  ver217 authored Apr 14, 2022
```
* fix reuse_fp16_shard

* disable test stm

* polish code
```
  a93a7d73
- [hotfix] fix auto tensor placement policy (#753) · 8f7ce94b
  ver217 authored Apr 14, 2022
  
  8f7ce94b
- [zero] refactor memstats_collector (#746) · 84c6700b
  HELSON authored Apr 14, 2022
  
  84c6700b
- [TP] allow layernorm without bias (#750) · b8899e09
  アマデウス authored Apr 14, 2022
  
  b8899e09
- [zero] use factory pattern for tensor_placement_policy (#752) · 3d7dc46d
  Jiarui Fang authored Apr 14, 2022
  
  3d7dc46d
13 Apr, 2022 6 commits
- fix prepare grads in sharded optim (#749) · 4b048a87
  ver217 authored Apr 13, 2022
  
  4b048a87
- fix initialize about zero · 09777254
  ver217 authored Apr 13, 2022
  
  09777254
- [zero] add tensor placement policies (#743) · e396bb71
  ver217 authored Apr 13, 2022
```
* add tensor placement policies

* polish comments

* polish comments

* update moe unit tests
```
  e396bb71
- [zero] refactor ShardedParamV2 for convenience (#742) · 22c4b88d
  HELSON authored Apr 13, 2022
  
  22c4b88d
- [utils] add synchronized cuda memory monitor (#740) · 340e59f9
  HELSON authored Apr 13, 2022
  
  340e59f9
- [hotfix] fix memory leak in backward of sharded model (#741) · e6212f56
  ver217 authored Apr 13, 2022
  
  e6212f56
12 Apr, 2022 9 commits
- [bug] fixed DDP compatibility with torch 1.8 (#739) · f4f42d4c
  Frank Lee authored Apr 13, 2022
  
  f4f42d4c
- [bug] fixed grad scaler compatibility with torch 1.8 (#735) · a4e91bc8
  Frank Lee authored Apr 12, 2022
  
  a4e91bc8
- [utils] correct cpu memory used and capacity in the context of multi-process (#726) · 53cb5848
  Jiarui Fang authored Apr 12, 2022
  
  53cb5848
- [hotfix] remove duplicated param register to stateful tensor manager (#728) · 7db3ccc7
  Jiarui Fang authored Apr 12, 2022
  
  7db3ccc7
- add video (#732) · 600e769a
  binmakeswell authored Apr 12, 2022
  
  600e769a
- [bug] removed zero installation requirements (#731) · a5c3f072
  Frank Lee authored Apr 12, 2022
  
  a5c3f072
- [moe] add checkpoint for moe zero test (#729) · b9b469ea
  HELSON authored Apr 12, 2022
  
  b9b469ea
- [doc] removed outdated installation command (#730) · 6f7d1362
  Frank Lee authored Apr 12, 2022
  
  6f7d1362
- [test] removed trivial outdated test · e88a498c
  FrankLeeeee authored Apr 12, 2022
  
  e88a498c