Commits · 22c4b88d56040e932697a74051f78da4e2971107 · OpenDAS / ColossalAI

13 Apr, 2022 1 commit
- [zero] refactor ShardedParamV2 for convenience (#742) · 22c4b88d
  HELSON authored Apr 13, 2022
  
  22c4b88d
12 Apr, 2022 1 commit
- [utils] correct cpu memory used and capacity in the context of multi-process (#726) · 53cb5848
  Jiarui Fang authored Apr 12, 2022
  
  53cb5848
11 Apr, 2022 1 commit
- [refactor] refactor the memory utils (#715) · 193dc8da
  Jiarui Fang authored Apr 11, 2022
  
  193dc8da
07 Apr, 2022 1 commit
- [zero] fix init bugs in zero context (#686) · d7ecaf36
  HELSON authored Apr 07, 2022
```
* adapt model weight initialization for methods in Pytorch nn.init
```
  d7ecaf36
01 Apr, 2022 1 commit
- [refactor] memory utils (#577) · e956d93a
  Jiarui Fang authored Apr 01, 2022
  
  e956d93a
31 Mar, 2022 1 commit
- [polish] rename col_attr -> colo_attr (#558) · 7675366f
  Jiarui Fang authored Mar 31, 2022
  
  7675366f
29 Mar, 2022 2 commits
- [zero] non model data tracing (#545) · 53b1b6e3
  Jiarui Fang authored Mar 29, 2022
  
  53b1b6e3
- [zero] polish ZeroInitContext (#540) · 1f90a3b1
  ver217 authored Mar 29, 2022
  
  1f90a3b1
28 Mar, 2022 1 commit
- [zero] refactor model data tracing (#537) · 705f5610
  Jiarui Fang authored Mar 28, 2022
  
  705f5610
25 Mar, 2022 3 commits
- [zero] refactor model data tracing (#522) · 8d8c5407
  Jiarui Fang authored Mar 25, 2022
  
  8d8c5407
- [test] fixed rerun_on_exception and adapted test cases (#487) · 3601b2ba
  Frank Lee authored Mar 25, 2022
  
  3601b2ba
- [zero] fix init device bug in zero init context unittest (#516) · 0bebda6e
  Jiarui Fang authored Mar 25, 2022
  
  0bebda6e
22 Mar, 2022 1 commit

[zero] polish sharded param name (#484) · b3348221

Jiarui Fang authored Mar 22, 2022

* [zero] polish sharded param name

* polish code

* polish

* polish code

* polish

* polsih

* polish

b3348221

18 Mar, 2022 2 commits
- [zero] Update initialize for ZeRO (#458) · a241f61b
  ver217 authored Mar 18, 2022
```
* polish code

* shard strategy receive pg in shard() / gather()

* update zero engine

* polish code
```
  a241f61b
- [test] optimized zero data parallel test (#452) · f27d801a
  Frank Lee authored Mar 18, 2022
  
  f27d801a
15 Mar, 2022 1 commit
- [polish] use GLOBAL_MODEL_DATA_TRACER (#417) · 56bb412e
  Jiarui Fang authored Mar 15, 2022
  
  56bb412e
14 Mar, 2022 2 commits
- [zero] memtracer to record cuda memory usage of model data and overall system (#395) · 21dc54e0
  Jiarui Fang authored Mar 14, 2022
  
  21dc54e0
- polish unit test · 54fd37f0
  ver217 authored Mar 14, 2022
  
  54fd37f0
11 Mar, 2022 7 commits
- [zero] zero init context collect numel of model (#375) · 6b600296
  Jiarui Fang authored Mar 10, 2022
  
  6b600296
- [zero] able to place params on cpu after zero init context (#365) · 44e4891f
  Jiarui Fang authored Mar 10, 2022
```
* place params on cpu after zero init context

* polish code
```
  44e4891f
- [zero] global model data memory tracer (#360) · ea287207
  Jiarui Fang authored Mar 10, 2022
  
  ea287207
- [zero] Update sharded model v2 using sharded param v2 (#323) · 13886716
  ver217 authored Mar 08, 2022
  
  13886716
- show pytest parameterize · dec24561
  jiaruifang authored Mar 08, 2022
  
  dec24561
- [zero] update zero context init with the updated test utils (#327) · 11bddb6e
  Jiarui Fang authored Mar 08, 2022
  
  11bddb6e
- [zero] zero init context (#321) · de0468c7
  Jiarui Fang authored Mar 07, 2022
```
* add zero init context

* add more flags for zero init context
fix bug of repeated converting param to ShardedParamV2

* polish code
```
  de0468c7