Commits · 1762ba14ab1c37e000c58d075315c04535d6221a · OpenDAS / ColossalAI

06 Apr, 2022 1 commit
- fix format (#563) · 1762ba14
  Ziyue Jiang authored Mar 31, 2022
  
  1762ba14
04 Apr, 2022 1 commit
- update GPT-2 experiment result (#666) · ce8a3eae
  Sze-qq authored Apr 04, 2022
  
  ce8a3eae
03 Apr, 2022 3 commits
- [hotfix] fix bugs for unsharded parameters when restore data (#664) · 17e73e62
  HELSON authored Apr 03, 2022
  
  17e73e62
- [hotfix] fix a bug in model data stats tracing (#655) · 0aab5230
  Jiarui Fang authored Apr 03, 2022
  
  0aab5230
- [refactor] pipeline, put runtime schedule into engine. (#627) · ade05a5d
  YuliangLiu0306 authored Apr 03, 2022
  
  ade05a5d
02 Apr, 2022 8 commits
- [hotfix] fix bugs in testing (#659) · e5d615ae
  HELSON authored Apr 02, 2022
```
* remove hybrid adam in test_moe_zero_optim

* fix activation checkpointing and its unitest
```
  e5d615ae
- Revert "[zero] polish init context (#645)" (#657) · 036404ca
  Jiarui Fang authored Apr 02, 2022
  
  036404ca
- fix bugs in CPU adam (#633) · b31daed4
  HELSON authored Apr 02, 2022
```
* add cpu adam counter for all cpu adam

* fixed updating error in adam kernel
```
  b31daed4
- [zero] fixed the activation offload (#647) · 1e2557e8
  LuGY authored Apr 02, 2022
```
* fixed the activation offload

* polish
```
  1e2557e8
- [hotfix] Raise messages for indivisible batch sizes with tensor parallelism (#622) · 828e4656
  Liang Bowen authored Apr 02, 2022
  
  828e4656
- [GitHub] Add prefix and label in issue template (#652) · e0f875a8
  binmakeswell authored Apr 02, 2022
  
  e0f875a8
- [zero] polish init context (#645) · 67b49282
  Jiarui Fang authored Apr 02, 2022
  
  67b49282
- polish checkpoint docstring (#637) · f5d3a9c2
  ver217 authored Apr 02, 2022
  
  f5d3a9c2
01 Apr, 2022 24 commits
- [zero] adapt zero for unsharded paramters (Optimizer part) (#601) · 055fbf5b
  HELSON authored Apr 01, 2022
  
  055fbf5b
- [NFC] polish colossalai/kernel/cuda_native/csrc/kernels/cuda_util.cu code stype (#625) · 229382c8
  KAIYUAN GAN authored Apr 01, 2022
  
  229382c8
- [model checkpoint] added unit tests for checkpoint save/load (#599) · 354b7954
  アマデウス authored Apr 01, 2022
  
  354b7954
- [model checkpoint] updated checkpoint hook (#598) · 28b515d6
  アマデウス authored Apr 01, 2022
  
  28b515d6
- [model checkpoint] updated saving/loading for 3d layers (#597) · 77ad24bf
  アマデウス authored Apr 01, 2022
  
  77ad24bf
- [model checkpoint] updated saving/loading for 2.5d layers (#596) · 93089ed7
  アマデウス authored Apr 01, 2022
  
  93089ed7
- [model checkpoint] updated communication ops for cpu tensors (#590) · 6302069c
  アマデウス authored Apr 01, 2022
  
  6302069c
- [model checkpoint] updated saving/loading for 1d layers (#594) · c50bfb80
  アマデウス authored Apr 01, 2022
  
  c50bfb80
- [model checkpoint] updated saving/loading for 2d layers (#595) · 7636d518
  アマデウス authored Apr 01, 2022
  
  7636d518
- [model checkpoint] reworked unified layers for ease of save/load states (#593) · cd13b638
  アマデウス authored Apr 01, 2022
  
  cd13b638
- [model checkpoint] updated checkpoint save/load utils (#592) · acae68eb
  アマデウス authored Apr 01, 2022
  
  acae68eb
- [TP] add assert for tp1d (#621) · 1c40ee87
  Ziyue Jiang authored Apr 01, 2022
  
  1c40ee87
- polish utils docstring (#620) · 369a288b
  ver217 authored Apr 01, 2022
  
  369a288b
- polish optimizer docstring (#619) · e619a651
  ver217 authored Apr 01, 2022
  
  e619a651
- polish moe docsrting (#618) · 8432dc70
  ver217 authored Apr 01, 2022
  
  8432dc70
- polish amp docstring (#616) · c5b488ed
  ver217 authored Apr 01, 2022
  
  c5b488ed
- update rst (#615) · f69507dd
  ver217 authored Apr 01, 2022
  
  f69507dd
- [zero] test zero tensor utils (#609) · 93f14d2a
  FredHuang99 authored Apr 01, 2022
  
  93f14d2a
- polish docstring of zero (#612) · 0ef8819c
  ver217 authored Apr 01, 2022
  
  0ef8819c
- [zero] add sampling time for memstats collector (#610) · 02b187c1
  LuGY authored Apr 01, 2022
  
  02b187c1
- [hotfix] fix sharded optim zero grad (#604) · 9bee1191
  ver217 authored Apr 01, 2022
```
* fix sharded optim zero grad

* polish comments
```
  9bee1191
- [model checkpoint] add gloo groups for cpu tensor communication (#589) · 297b8baa
  アマデウス authored Apr 01, 2022
  
  297b8baa
- moved ensure_path_exists to utils.common (#591) · 54e688b6
  アマデウス authored Apr 01, 2022
  
  54e688b6
- [refactor] memory utils (#577) · e956d93a
  Jiarui Fang authored Apr 01, 2022
  
  e956d93a
31 Mar, 2022 3 commits
- [hotfix] add hybrid adam to __init__ (#584) · 104cbbb3
  ver217 authored Mar 31, 2022
  
  104cbbb3
- [zero] adapt zero for unsharded parameters (#561) · e6d50ec1
  HELSON authored Mar 31, 2022
```
* support existing sharded and unsharded parameters in zero

* add unitest for moe-zero model init

* polish moe gradient handler
```
  e6d50ec1
- [model zoo] add activation offload for gpt model (#582) · 13ed4b64
  LuGY authored Mar 31, 2022
  
  13ed4b64