Commits · b09adff724c2bbded1c71cc51a707f736a0e2899 · OpenDAS / ColossalAI

28 Jan, 2023 1 commit
- [gemini] update ddp strict mode (#2518) · 707b11d4
  HELSON authored Jan 28, 2023
```
* [zero] add strict ddp mode for chunk init

* [gemini] update gpt example
```
  707b11d4
18 Oct, 2022 1 commit
- [zero] add chunk init function for users (#1729) · f69f9bf2
  HELSON authored Oct 18, 2022
```
* add chunk manager init function

* fix unit tests

* add comment

* add flush=True
```
  f69f9bf2
09 Oct, 2022 1 commit
- [feature] A new ZeRO implementation (#1644) · b28991dd
  HELSON authored Oct 09, 2022
  
  b28991dd
26 Sep, 2022 1 commit
- Revert "[feature] new zero implementation (#1623)" (#1643) · c5d39215
  Jiarui Fang authored Sep 26, 2022
```
This reverts commit 5be118f4.
```
  c5d39215
24 Sep, 2022 1 commit
- [feature] new zero implementation (#1623) · 5be118f4
  HELSON authored Sep 24, 2022
  
  5be118f4
02 Aug, 2022 1 commit
- [zero] alleviate memory usage in ZeRODDP state_dict (#1398) · 4e98e938
  HELSON authored Aug 02, 2022
  
  4e98e938
28 Jul, 2022 1 commit
- [DDP] test ddp state dict uses more strict threshold (#1382) · 7d5d628e
  ver217 authored Jul 28, 2022
  
  7d5d628e
26 Jul, 2022 1 commit
- [colotensor] use cpu memory to store state_dict (#1367) · 87775a06
  HELSON authored Jul 26, 2022
  
  87775a06
18 Jul, 2022 1 commit

ver217 authored Jul 18, 2022

* process group supports getting ranks in group

* chunk mgr receives a process group

* update unit test

* fix unit tests

0c51ff2c

08 Jul, 2022 1 commit
- [tensor] fix some unittests (#1234) · 3b500984
  Jiarui Fang authored Jul 08, 2022
  
  3b500984
04 Jul, 2022 1 commit
- [refactor] remove gpc dependency in colotensor's _ops (#1189) · 060b917d
  Jiarui Fang authored Jul 04, 2022
  
  060b917d
29 Jun, 2022 2 commits
- [refactor] move chunk and chunkmgr to directory gemini (#1182) · 372f7914
  Jiarui Fang authored Jun 29, 2022
  
  372f7914
- [ddp] ColoDDP uses bucket all-reduce (#1177) · 6b2f2ab9
  ver217 authored Jun 29, 2022
```
* add reducer

* update colo ddp with reducer

* polish unit test

* polish unit test
```
  6b2f2ab9
21 Jun, 2022 1 commit

ver217 authored Jun 21, 2022

* ColoDDP supports overwriting default process group

* rename ColoDDPV2 to ZeroDDP

* add docstr for ZeroDDP

* polish docstr

8106d7b8

ver217 authored Jun 20, 2022

* add save/load state dict for ColoDDP

* add unit test

* refactor unit test folder

* polish unit test

* rename unit test

d2690264