Commits · b09adff724c2bbded1c71cc51a707f736a0e2899 · OpenDAS / ColossalAI

20 Mar, 2023 1 commit

[test] fixed torchrec model test (#3167) · 1ad3a636

Frank Lee authored Mar 20, 2023

* [test] fixed torchrec model test

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code

1ad3a636

17 Feb, 2023 1 commit
- [hotfix] add correct device for fake_param (#2796) · 56ddc9ca
  HELSON authored Feb 17, 2023
  
  56ddc9ca
13 Feb, 2023 1 commit
- [gemini] add fake_release_chunk for keep-gathered chunk in the inference mode (#2671) · 8213f89f
  HELSON authored Feb 13, 2023
  
  8213f89f
28 Jan, 2023 1 commit
- [gemini] update ddp strict mode (#2518) · 707b11d4
  HELSON authored Jan 28, 2023
```
* [zero] add strict ddp mode for chunk init

* [gemini] update gpt example
```
  707b11d4
11 Jan, 2023 2 commits
- [zero] fix state_dict and load_state_dict for ddp ignored parameters (#2443) · 5521af78
  HELSON authored Jan 11, 2023
```
* [ddp] add is_ddp_ignored

[ddp] rename to is_ddp_ignored

* [zero] fix state_dict and load_state_dict

* fix bugs

* [zero] update unit test for ZeroDDP
```
  5521af78
- [zero] add inference mode and its unit test (#2418) · bb4e9a31
  HELSON authored Jan 11, 2023
  
  bb4e9a31
09 Jan, 2023 1 commit
- [polish] polish code for get_static_torch_model (#2405) · ea13a201
  HELSON authored Jan 09, 2023
```
* [gemini] polish code

* [testing] remove code

* [gemini] make more robust
```
  ea13a201
06 Jan, 2023 1 commit
- [gemini] add get static torch model (#2356) · 48d33b1b
  HELSON authored Jan 06, 2023
  
  48d33b1b
26 Dec, 2022 2 commits

[testing] add beit model for unit testings (#2196) · a3100bd5
HELSON authored Dec 26, 2022
```
* [testing] add beit model

* [beit] fix bugs

* [beit] fix bugs

* [testing] fix bugs
```
a3100bd5

[zero] fix error for BEiT models (#2169) · 24586599

HELSON authored Dec 26, 2022

* [zero] fix error for BEiT models

* [ColoParameter] add unpack operation for tuple arguments

* fix bugs

* fix chunkv2 unit testing

* add assertion for gradient state

24586599

22 Dec, 2022 1 commit
- [example] add palm pytorch version (#2172) · 27327a4c
  Jiarui Fang authored Dec 22, 2022
  
  27327a4c
20 Dec, 2022 1 commit
- [Gemini] GeminiDPP convert to PyTorch Module. (#2151) · 2827f418
  Jiarui Fang authored Dec 20, 2022
  
  2827f418
13 Dec, 2022 6 commits
- [Gemini] update API of the chunkmemstatscollector. (#2129) · c89c66a8
  Jiarui Fang authored Dec 14, 2022
  
  c89c66a8
- [Gemini] update the non model data record method in runtime memory tracer (#2128) · 2938edf4
  Jiarui Fang authored Dec 13, 2022
  
  2938edf4
- [Gemini] test step-tensor mapping using repeated_computed_layers.py (#2127) · deee317b
  Jiarui Fang authored Dec 13, 2022
  
  deee317b
- [Gemini] update non model data calculation method (#2126) · 8fac8376
  Jiarui Fang authored Dec 13, 2022
  
  8fac8376
- [Gemini] hotfix the unittest bugs (#2125) · 5efda697
  Jiarui Fang authored Dec 13, 2022
  
  5efda697
- [Gemini] mapping of preop timestep and param (#2124) · 05bb28aa
  Jiarui Fang authored Dec 13, 2022
  
  05bb28aa
12 Dec, 2022 3 commits
- [Gemini] chunk init using runtime visited param order (#2115) · 9214d1fe
  Jiarui Fang authored Dec 12, 2022
  
  9214d1fe
- [NFC] update chunk manager API (#2119) · e5aa8333
  Jiarui Fang authored Dec 12, 2022
  
  e5aa8333
- [NFC] polish comments for Chunk class (#2116) · e99edfcb
  Jiarui Fang authored Dec 12, 2022
  
  e99edfcb
09 Dec, 2022 2 commits
- [zero] add L2 gradient clipping for ZeRO (#2112) · 63fbba3c
  HELSON authored Dec 09, 2022
```
* [zero] add L2 gradient clipping

* [testing] add MlpModel

* [zero] add unit test for grad clipping

* fix atol
```
  63fbba3c
- [gemini] get the param visited order during runtime (#2108) · 70a85569
  Jiarui Fang authored Dec 09, 2022
  
  70a85569
07 Dec, 2022 2 commits
- [Gemini] gemini use the runtime memory tracer (RMT) (#2099) · 85efb7ac
  Jiarui Fang authored Dec 07, 2022
  
  85efb7ac
- [Gemini] remove eval in gemini unittests! (#2092) · 97824232
  Jiarui Fang authored Dec 07, 2022
  
  97824232
06 Dec, 2022 2 commits
- [Gemini] remove GLOBAL_MODEL_DATA_TRACER (#2091) · 1fca5d79
  Jiarui Fang authored Dec 06, 2022
  
  1fca5d79
- [Gemini] use MemStats in Runtime Memory tracer (#2088) · 25abae6d
  Jiarui Fang authored Dec 06, 2022
  
  25abae6d
05 Dec, 2022 6 commits
- [Gemini] polish runtime tracer tests (#2077) · 4f21c9e8
  Jiarui Fang authored Dec 05, 2022
  
  4f21c9e8
- [Gemini] rename hooks related to runtime mem tracer (#2076) · a7adad9c
  Jiarui Fang authored Dec 05, 2022
  
  a7adad9c
- [Gemini] add albert in test models. (#2075) · 40b7d55b
  Jiarui Fang authored Dec 05, 2022
  
  40b7d55b
- [test] bert test in non-distributed way (#2074) · 616ed91e
  Jiarui Fang authored Dec 05, 2022
  
  616ed91e
- [Gemini] rename ParamTracerWrapper -> RuntimeMemTracer (#2073) · 223332ff
  Jiarui Fang authored Dec 05, 2022
  
  223332ff
- [Gemini] remove not used MemtracerWrapper (#2072) · 9f828ef3
  Jiarui Fang authored Dec 05, 2022
  
  9f828ef3
02 Dec, 2022 1 commit
- [Gemini] fix grad unreleased issue and param recovery issue (#2052) · 38ea4ba1
  Zihao authored Dec 02, 2022
  
  38ea4ba1
30 Nov, 2022 6 commits
- [gemini] fix init bugs for modules (#2047) · f6178728
  HELSON authored Nov 30, 2022
```
* [gemini] fix init bugs for modules

* fix bugs
```
  f6178728
- [Gemini] free and allocate cuda memory by tensor.storage, add grad hook (#2040) · 6a9158f1
  Zihao authored Nov 30, 2022
  
  6a9158f1
- [test] align model name with the file name. (#2045) · 1e885329
  Jiarui Fang authored Nov 30, 2022
  
  1e885329
- [hotfix] hotfix Gemini for no leaf modules bug (#2043) · 31c64402
  Jiarui Fang authored Nov 30, 2022
  
  31c64402
- [zero] fix testing parameters (#2042) · 384cd263
  HELSON authored Nov 30, 2022
  
  384cd263
- [zero] fix unit-tests (#2039) · 17a3c685
  HELSON authored Nov 30, 2022
  
  17a3c685