Commits · 7636d518e1033b741e7346d21fb551b55a4384ad · OpenDAS / ColossalAI

"tests/test_ops/test_embedding_bag_tp.py" did not exist on "ae861519687b09934efe431f951b052f7b811b11"

01 Apr, 2022 16 commits
- [model checkpoint] updated saving/loading for 2d layers (#595) · 7636d518
  アマデウス authored Apr 01, 2022
  
  7636d518
- [model checkpoint] reworked unified layers for ease of save/load states (#593) · cd13b638
  アマデウス authored Apr 01, 2022
  
  cd13b638
- [model checkpoint] updated checkpoint save/load utils (#592) · acae68eb
  アマデウス authored Apr 01, 2022
  
  acae68eb
- [TP] add assert for tp1d (#621) · 1c40ee87
  Ziyue Jiang authored Apr 01, 2022
  
  1c40ee87
- polish utils docstring (#620) · 369a288b
  ver217 authored Apr 01, 2022
  
  369a288b
- polish optimizer docstring (#619) · e619a651
  ver217 authored Apr 01, 2022
  
  e619a651
- polish moe docsrting (#618) · 8432dc70
  ver217 authored Apr 01, 2022
  
  8432dc70
- polish amp docstring (#616) · c5b488ed
  ver217 authored Apr 01, 2022
  
  c5b488ed
- update rst (#615) · f69507dd
  ver217 authored Apr 01, 2022
  
  f69507dd
- [zero] test zero tensor utils (#609) · 93f14d2a
  FredHuang99 authored Apr 01, 2022
  
  93f14d2a
- polish docstring of zero (#612) · 0ef8819c
  ver217 authored Apr 01, 2022
  
  0ef8819c
- [zero] add sampling time for memstats collector (#610) · 02b187c1
  LuGY authored Apr 01, 2022
  
  02b187c1
- [hotfix] fix sharded optim zero grad (#604) · 9bee1191
  ver217 authored Apr 01, 2022
```
* fix sharded optim zero grad

* polish comments
```
  9bee1191
- [model checkpoint] add gloo groups for cpu tensor communication (#589) · 297b8baa
  アマデウス authored Apr 01, 2022
  
  297b8baa
- moved ensure_path_exists to utils.common (#591) · 54e688b6
  アマデウス authored Apr 01, 2022
  
  54e688b6
- [refactor] memory utils (#577) · e956d93a
  Jiarui Fang authored Apr 01, 2022
  
  e956d93a
31 Mar, 2022 9 commits
- [hotfix] add hybrid adam to __init__ (#584) · 104cbbb3
  ver217 authored Mar 31, 2022
  
  104cbbb3
- [zero] adapt zero for unsharded parameters (#561) · e6d50ec1
  HELSON authored Mar 31, 2022
```
* support existing sharded and unsharded parameters in zero

* add unitest for moe-zero model init

* polish moe gradient handler
```
  e6d50ec1
- [model zoo] add activation offload for gpt model (#582) · 13ed4b64
  LuGY authored Mar 31, 2022
  
  13ed4b64
- update code format · 46c9ba33
  Wesley authored Mar 31, 2022
  
  46c9ba33
- fix parallel_input flag for Linear1D_Col gather_output · 666cfd09
  Wesley authored Mar 31, 2022
  
  666cfd09
- [tool] create .clang-format for pre-commit (#578) · a9f778f1
  BoxiangW authored Mar 31, 2022
```
Change the clang-format style to google style
```
  a9f778f1
- [zero] trace states of fp16/32 grad and fp32 param (#571) · 7c6c427d
  ver217 authored Mar 31, 2022
  
  7c6c427d
- [polish] rename col_attr -> colo_attr (#558) · 7675366f
  Jiarui Fang authored Mar 31, 2022
  
  7675366f
- html refactor (#555) · 2c45efc3
  Liang Bowen authored Mar 31, 2022
  
  2c45efc3
30 Mar, 2022 8 commits
- [utils] update colo tensor moving APIs (#553) · d1211148
  Jiarui Fang authored Mar 30, 2022
  
  d1211148
- [docs] updatad docs of hybrid adam and cpu adam (#552) · c44d7970
  LuGY authored Mar 30, 2022
  
  c44d7970
- [zero] hijack p.grad in sharded model (#554) · 014bac0c
  ver217 authored Mar 30, 2022
```
* hijack p.grad in sharded model

* polish comments

* polish comments
```
  014bac0c
- [zero] label state for param fp16 and grad (#551) · f552b112
  Jiarui Fang authored Mar 30, 2022
  
  f552b112
- Automated submodule synchronization (#501) · 92f42248
  github-actions[bot] authored Mar 30, 2022
  
  92f42248
- [zero] add stateful tensor (#549) · 214da761
  Jiarui Fang authored Mar 30, 2022
  
  214da761
- [zero] dump memory stats for sharded model (#548) · 107b99dd
  Jiarui Fang authored Mar 30, 2022
  
  107b99dd
- [TP] Add gather_out arg to Linear (#541) · 763dc325
  Ziyue Jiang authored Mar 30, 2022
  
  763dc325
29 Mar, 2022 7 commits
- [zero] add zero context manager to change config during initialization (#546) · 8c90d4df
  HELSON authored Mar 29, 2022
  
  8c90d4df
- Refactored docstring to google style · ec5086c4
  Liang Bowen authored Mar 25, 2022
  
  ec5086c4
- [zero] non model data tracing (#545) · 53b1b6e3
  Jiarui Fang authored Mar 29, 2022
  
  53b1b6e3
- [profiler] add MemProfiler (#356) · 73d36618
  Jie Zhu authored Mar 29, 2022
```
* add memory trainer hook

* fix bug

* add memory trainer hook

* fix import bug

* fix import bug

* add trainer hook

* fix #370 git log bug

* modify `to_tensorboard` function to support better output

* remove useless output

* change the name of `MemProfiler`

* complete memory profiler

* replace error with warning

* finish trainer hook

* modify interface of MemProfiler

* modify `__init__.py` in profiler

* remove unnecessary pass statement

* add usage to doc string

* add usage to trainer hook

* new location to store temp data file
```
  73d36618
- [zero] optimize grad offload (#539) · fb841dd5
  ver217 authored Mar 29, 2022
```
* optimize grad offload

* polish code

* polish code
```
  fb841dd5
- [logging] polish logger format (#543) · 7d81b5b4
  Jiarui Fang authored Mar 29, 2022
  
  7d81b5b4
- [zero] polish ZeroInitContext (#540) · 1f90a3b1
  ver217 authored Mar 29, 2022
  
  1f90a3b1