Commits · 9caa8b6481b3ed8d5555502c7671dcd2493c0da6 · OpenDAS / ColossalAI

"docs/vscode:/vscode.git/clone" did not exist on "416a50dbd713edab5ccb39aaf6dd1aecb0520e09"

18 Mar, 2022 2 commits
- [zero] Update initialize for ZeRO (#458) · a241f61b
  ver217 authored Mar 18, 2022
```
* polish code

* shard strategy receive pg in shard() / gather()

* update zero engine

* polish code
```
  a241f61b
- [test] optimized zero data parallel test (#452) · f27d801a
  Frank Lee authored Mar 18, 2022
  
  f27d801a
15 Mar, 2022 1 commit
- [polish] use GLOBAL_MODEL_DATA_TRACER (#417) · 56bb412e
  Jiarui Fang authored Mar 15, 2022
  
  56bb412e
14 Mar, 2022 2 commits
- [zero] memtracer to record cuda memory usage of model data and overall system (#395) · 21dc54e0
  Jiarui Fang authored Mar 14, 2022
  
  21dc54e0
- polish unit test · 54fd37f0
  ver217 authored Mar 14, 2022
  
  54fd37f0
11 Mar, 2022 7 commits
- [zero] zero init context collect numel of model (#375) · 6b600296
  Jiarui Fang authored Mar 10, 2022
  
  6b600296
- [zero] able to place params on cpu after zero init context (#365) · 44e4891f
  Jiarui Fang authored Mar 10, 2022
```
* place params on cpu after zero init context

* polish code
```
  44e4891f
- [zero] global model data memory tracer (#360) · ea287207
  Jiarui Fang authored Mar 10, 2022
  
  ea287207
- [zero] Update sharded model v2 using sharded param v2 (#323) · 13886716
  ver217 authored Mar 08, 2022
  
  13886716
- show pytest parameterize · dec24561
  jiaruifang authored Mar 08, 2022
  
  dec24561
- [zero] update zero context init with the updated test utils (#327) · 11bddb6e
  Jiarui Fang authored Mar 08, 2022
  
  11bddb6e
- [zero] zero init context (#321) · de0468c7
  Jiarui Fang authored Mar 07, 2022
```
* add zero init context

* add more flags for zero init context
fix bug of repeated converting param to ShardedParamV2

* polish code
```
  de0468c7