Commits · 5682e6d34645db87d6656a6a1f90f8425cb0420e · OpenDAS / ColossalAI

26 Dec, 2022 1 commit

[zero] fix error for BEiT models (#2169) · 24586599

HELSON authored Dec 26, 2022

* [zero] fix error for BEiT models

* [ColoParameter] add unpack operation for tuple arguments

* fix bugs

* fix chunkv2 unit testing

* add assertion for gradient state

24586599

22 Dec, 2022 1 commit
- [example] add palm pytorch version (#2172) · 27327a4c
  Jiarui Fang authored Dec 22, 2022
  
  27327a4c
20 Dec, 2022 1 commit
- [Gemini] GeminiDPP convert to PyTorch Module. (#2151) · 2827f418
  Jiarui Fang authored Dec 20, 2022
  
  2827f418
13 Dec, 2022 6 commits
- [Gemini] update API of the chunkmemstatscollector. (#2129) · c89c66a8
  Jiarui Fang authored Dec 14, 2022
  
  c89c66a8
- [Gemini] update the non model data record method in runtime memory tracer (#2128) · 2938edf4
  Jiarui Fang authored Dec 13, 2022
  
  2938edf4
- [Gemini] test step-tensor mapping using repeated_computed_layers.py (#2127) · deee317b
  Jiarui Fang authored Dec 13, 2022
  
  deee317b
- [Gemini] update non model data calculation method (#2126) · 8fac8376
  Jiarui Fang authored Dec 13, 2022
  
  8fac8376
- [Gemini] hotfix the unittest bugs (#2125) · 5efda697
  Jiarui Fang authored Dec 13, 2022
  
  5efda697
- [Gemini] mapping of preop timestep and param (#2124) · 05bb28aa
  Jiarui Fang authored Dec 13, 2022
  
  05bb28aa
12 Dec, 2022 3 commits
- [Gemini] chunk init using runtime visited param order (#2115) · 9214d1fe
  Jiarui Fang authored Dec 12, 2022
  
  9214d1fe
- [NFC] update chunk manager API (#2119) · e5aa8333
  Jiarui Fang authored Dec 12, 2022
  
  e5aa8333
- [NFC] polish comments for Chunk class (#2116) · e99edfcb
  Jiarui Fang authored Dec 12, 2022
  
  e99edfcb
09 Dec, 2022 2 commits
- [zero] add L2 gradient clipping for ZeRO (#2112) · 63fbba3c
  HELSON authored Dec 09, 2022
```
* [zero] add L2 gradient clipping

* [testing] add MlpModel

* [zero] add unit test for grad clipping

* fix atol
```
  63fbba3c
- [gemini] get the param visited order during runtime (#2108) · 70a85569
  Jiarui Fang authored Dec 09, 2022
  
  70a85569
07 Dec, 2022 2 commits
- [Gemini] gemini use the runtime memory tracer (RMT) (#2099) · 85efb7ac
  Jiarui Fang authored Dec 07, 2022
  
  85efb7ac
- [Gemini] remove eval in gemini unittests! (#2092) · 97824232
  Jiarui Fang authored Dec 07, 2022
  
  97824232
06 Dec, 2022 2 commits
- [Gemini] remove GLOBAL_MODEL_DATA_TRACER (#2091) · 1fca5d79
  Jiarui Fang authored Dec 06, 2022
  
  1fca5d79
- [Gemini] use MemStats in Runtime Memory tracer (#2088) · 25abae6d
  Jiarui Fang authored Dec 06, 2022
  
  25abae6d
05 Dec, 2022 6 commits
- [Gemini] polish runtime tracer tests (#2077) · 4f21c9e8
  Jiarui Fang authored Dec 05, 2022
  
  4f21c9e8
- [Gemini] rename hooks related to runtime mem tracer (#2076) · a7adad9c
  Jiarui Fang authored Dec 05, 2022
  
  a7adad9c
- [Gemini] add albert in test models. (#2075) · 40b7d55b
  Jiarui Fang authored Dec 05, 2022
  
  40b7d55b
- [test] bert test in non-distributed way (#2074) · 616ed91e
  Jiarui Fang authored Dec 05, 2022
  
  616ed91e
- [Gemini] rename ParamTracerWrapper -> RuntimeMemTracer (#2073) · 223332ff
  Jiarui Fang authored Dec 05, 2022
  
  223332ff
- [Gemini] remove not used MemtracerWrapper (#2072) · 9f828ef3
  Jiarui Fang authored Dec 05, 2022
  
  9f828ef3
02 Dec, 2022 1 commit
- [Gemini] fix grad unreleased issue and param recovery issue (#2052) · 38ea4ba1
  Zihao authored Dec 02, 2022
  
  38ea4ba1
30 Nov, 2022 6 commits
- [gemini] fix init bugs for modules (#2047) · f6178728
  HELSON authored Nov 30, 2022
```
* [gemini] fix init bugs for modules

* fix bugs
```
  f6178728
- [Gemini] free and allocate cuda memory by tensor.storage, add grad hook (#2040) · 6a9158f1
  Zihao authored Nov 30, 2022
  
  6a9158f1
- [test] align model name with the file name. (#2045) · 1e885329
  Jiarui Fang authored Nov 30, 2022
  
  1e885329
- [hotfix] hotfix Gemini for no leaf modules bug (#2043) · 31c64402
  Jiarui Fang authored Nov 30, 2022
  
  31c64402
- [zero] fix testing parameters (#2042) · 384cd263
  HELSON authored Nov 30, 2022
  
  384cd263
- [zero] fix unit-tests (#2039) · 17a3c685
  HELSON authored Nov 30, 2022
  
  17a3c685
29 Nov, 2022 4 commits
- [Gemini] more tests for Gemini (#2038) · eb7742a4
  Jiarui Fang authored Nov 29, 2022
```
* [Gemini] more tests for Gemini

* polish code
```
  eb7742a4
- [testing] fix testing models (#2036) · 537e1817
  HELSON authored Nov 29, 2022
```
* [testing] fix testing models

* roll back
```
  537e1817
- [hotfix] add bert test for gemini fwd bwd (#2035) · 96134e7b
  Jiarui Fang authored Nov 29, 2022
  
  96134e7b
- [Gemini] more rigorous unit tests for run_fwd_bwd (#2034) · 28aa9a42
  Jiarui Fang authored Nov 29, 2022
  
  28aa9a42
26 Nov, 2022 1 commit
- [Gemini] paramWrapper paramTracerHook unitest (#2030) · 95c4532f
  Zihao authored Nov 26, 2022
  
  95c4532f
25 Nov, 2022 1 commit
- [Gemini] patch for supporting orch.add_ function for ColoTensor (#2003) · 8daf1b4d
  Jiarui Fang authored Nov 25, 2022
  
  8daf1b4d
24 Nov, 2022 3 commits
- [Gemini] add unitests to check gemini correctness (#2015) · 2e9cbfca
  Jiarui Fang authored Nov 24, 2022
  
  2e9cbfca
- [hotfix] revert bug PRs (#2016) · 0b0d8f9e
  Jiarui Fang authored Nov 24, 2022
  
  0b0d8f9e
- [Gemini] param_tracer_wrapper and test case (#2009) · 0160a62a
  Zihao authored Nov 24, 2022
  
  0160a62a