Commits · 73d36618a6263b3e0ff3178a4e7dfaaec60655b0 · OpenDAS / ColossalAI

29 Mar, 2022 1 commit
- [zero] polish ZeroInitContext (#540) · 1f90a3b1
  ver217 authored Mar 29, 2022
  
  1f90a3b1
28 Mar, 2022 1 commit
- [zero] refactor model data tracing (#537) · 705f5610
  Jiarui Fang authored Mar 28, 2022
  
  705f5610
25 Mar, 2022 3 commits
- [zero] refactor model data tracing (#522) · 8d8c5407
  Jiarui Fang authored Mar 25, 2022
  
  8d8c5407
- [test] fixed rerun_on_exception and adapted test cases (#487) · 3601b2ba
  Frank Lee authored Mar 25, 2022
  
  3601b2ba
- [zero] fix init device bug in zero init context unittest (#516) · 0bebda6e
  Jiarui Fang authored Mar 25, 2022
  
  0bebda6e
22 Mar, 2022 1 commit

[zero] polish sharded param name (#484) · b3348221

Jiarui Fang authored Mar 22, 2022

* [zero] polish sharded param name

* polish code

* polish

* polish code

* polish

* polsih

* polish

b3348221

18 Mar, 2022 2 commits
- [zero] Update initialize for ZeRO (#458) · a241f61b
  ver217 authored Mar 18, 2022
```
* polish code

* shard strategy receive pg in shard() / gather()

* update zero engine

* polish code
```
  a241f61b
- [test] optimized zero data parallel test (#452) · f27d801a
  Frank Lee authored Mar 18, 2022
  
  f27d801a
15 Mar, 2022 1 commit
- [polish] use GLOBAL_MODEL_DATA_TRACER (#417) · 56bb412e
  Jiarui Fang authored Mar 15, 2022
  
  56bb412e
14 Mar, 2022 2 commits
- [zero] memtracer to record cuda memory usage of model data and overall system (#395) · 21dc54e0
  Jiarui Fang authored Mar 14, 2022
  
  21dc54e0
- polish unit test · 54fd37f0
  ver217 authored Mar 14, 2022
  
  54fd37f0
11 Mar, 2022 7 commits
- [zero] zero init context collect numel of model (#375) · 6b600296
  Jiarui Fang authored Mar 10, 2022
  
  6b600296
- [zero] able to place params on cpu after zero init context (#365) · 44e4891f
  Jiarui Fang authored Mar 10, 2022
```
* place params on cpu after zero init context

* polish code
```
  44e4891f
- [zero] global model data memory tracer (#360) · ea287207
  Jiarui Fang authored Mar 10, 2022
  
  ea287207
- [zero] Update sharded model v2 using sharded param v2 (#323) · 13886716
  ver217 authored Mar 08, 2022
  
  13886716
- show pytest parameterize · dec24561
  jiaruifang authored Mar 08, 2022
  
  dec24561
- [zero] update zero context init with the updated test utils (#327) · 11bddb6e
  Jiarui Fang authored Mar 08, 2022
  
  11bddb6e
- [zero] zero init context (#321) · de0468c7
  Jiarui Fang authored Mar 07, 2022
```
* add zero init context

* add more flags for zero init context
fix bug of repeated converting param to ShardedParamV2

* polish code
```
  de0468c7