Commits · 036404ca8a564d17446b59c09ba21ceffd0608f6 · OpenDAS / ColossalAI

02 Apr, 2022 7 commits
- Revert "[zero] polish init context (#645)" (#657) · 036404ca
  Jiarui Fang authored Apr 02, 2022
  
  036404ca
- fix bugs in CPU adam (#633) · b31daed4
  HELSON authored Apr 02, 2022
```
* add cpu adam counter for all cpu adam

* fixed updating error in adam kernel
```
  b31daed4
- [zero] fixed the activation offload (#647) · 1e2557e8
  LuGY authored Apr 02, 2022
```
* fixed the activation offload

* polish
```
  1e2557e8
- [hotfix] Raise messages for indivisible batch sizes with tensor parallelism (#622) · 828e4656
  Liang Bowen authored Apr 02, 2022
  
  828e4656
- [GitHub] Add prefix and label in issue template (#652) · e0f875a8
  binmakeswell authored Apr 02, 2022
  
  e0f875a8
- [zero] polish init context (#645) · 67b49282
  Jiarui Fang authored Apr 02, 2022
  
  67b49282
- polish checkpoint docstring (#637) · f5d3a9c2
  ver217 authored Apr 02, 2022
  
  f5d3a9c2
01 Apr, 2022 24 commits
- [zero] adapt zero for unsharded paramters (Optimizer part) (#601) · 055fbf5b
  HELSON authored Apr 01, 2022
  
  055fbf5b
- [NFC] polish colossalai/kernel/cuda_native/csrc/kernels/cuda_util.cu code stype (#625) · 229382c8
  KAIYUAN GAN authored Apr 01, 2022
  
  229382c8
- [model checkpoint] added unit tests for checkpoint save/load (#599) · 354b7954
  アマデウス authored Apr 01, 2022
  
  354b7954
- [model checkpoint] updated checkpoint hook (#598) · 28b515d6
  アマデウス authored Apr 01, 2022
  
  28b515d6
- [model checkpoint] updated saving/loading for 3d layers (#597) · 77ad24bf
  アマデウス authored Apr 01, 2022
  
  77ad24bf
- [model checkpoint] updated saving/loading for 2.5d layers (#596) · 93089ed7
  アマデウス authored Apr 01, 2022
  
  93089ed7
- [model checkpoint] updated communication ops for cpu tensors (#590) · 6302069c
  アマデウス authored Apr 01, 2022
  
  6302069c
- [model checkpoint] updated saving/loading for 1d layers (#594) · c50bfb80
  アマデウス authored Apr 01, 2022
  
  c50bfb80
- [model checkpoint] updated saving/loading for 2d layers (#595) · 7636d518
  アマデウス authored Apr 01, 2022
  
  7636d518
- [model checkpoint] reworked unified layers for ease of save/load states (#593) · cd13b638
  アマデウス authored Apr 01, 2022
  
  cd13b638
- [model checkpoint] updated checkpoint save/load utils (#592) · acae68eb
  アマデウス authored Apr 01, 2022
  
  acae68eb
- [TP] add assert for tp1d (#621) · 1c40ee87
  Ziyue Jiang authored Apr 01, 2022
  
  1c40ee87
- polish utils docstring (#620) · 369a288b
  ver217 authored Apr 01, 2022
  
  369a288b
- polish optimizer docstring (#619) · e619a651
  ver217 authored Apr 01, 2022
  
  e619a651
- polish moe docsrting (#618) · 8432dc70
  ver217 authored Apr 01, 2022
  
  8432dc70
- polish amp docstring (#616) · c5b488ed
  ver217 authored Apr 01, 2022
  
  c5b488ed
- update rst (#615) · f69507dd
  ver217 authored Apr 01, 2022
  
  f69507dd
- [zero] test zero tensor utils (#609) · 93f14d2a
  FredHuang99 authored Apr 01, 2022
  
  93f14d2a
- polish docstring of zero (#612) · 0ef8819c
  ver217 authored Apr 01, 2022
  
  0ef8819c
- [zero] add sampling time for memstats collector (#610) · 02b187c1
  LuGY authored Apr 01, 2022
  
  02b187c1
- [hotfix] fix sharded optim zero grad (#604) · 9bee1191
  ver217 authored Apr 01, 2022
```
* fix sharded optim zero grad

* polish comments
```
  9bee1191
- [model checkpoint] add gloo groups for cpu tensor communication (#589) · 297b8baa
  アマデウス authored Apr 01, 2022
  
  297b8baa
- moved ensure_path_exists to utils.common (#591) · 54e688b6
  アマデウス authored Apr 01, 2022
  
  54e688b6
- [refactor] memory utils (#577) · e956d93a
  Jiarui Fang authored Apr 01, 2022
  
  e956d93a
31 Mar, 2022 9 commits
- [hotfix] add hybrid adam to __init__ (#584) · 104cbbb3
  ver217 authored Mar 31, 2022
  
  104cbbb3
- [zero] adapt zero for unsharded parameters (#561) · e6d50ec1
  HELSON authored Mar 31, 2022
```
* support existing sharded and unsharded parameters in zero

* add unitest for moe-zero model init

* polish moe gradient handler
```
  e6d50ec1
- [model zoo] add activation offload for gpt model (#582) · 13ed4b64
  LuGY authored Mar 31, 2022
  
  13ed4b64
- update code format · 46c9ba33
  Wesley authored Mar 31, 2022
  
  46c9ba33
- fix parallel_input flag for Linear1D_Col gather_output · 666cfd09
  Wesley authored Mar 31, 2022
  
  666cfd09
- [tool] create .clang-format for pre-commit (#578) · a9f778f1
  BoxiangW authored Mar 31, 2022
```
Change the clang-format style to google style
```
  a9f778f1
- [zero] trace states of fp16/32 grad and fp32 param (#571) · 7c6c427d
  ver217 authored Mar 31, 2022
  
  7c6c427d
- [polish] rename col_attr -> colo_attr (#558) · 7675366f
  Jiarui Fang authored Mar 31, 2022
  
  7675366f
- html refactor (#555) · 2c45efc3
  Liang Bowen authored Mar 31, 2022
  
  2c45efc3