Commits · 72e40c740174fe2b3f8c9999c5e1f77d8dca3018 · OpenDAS / FastMoE

"docs/vscode:/vscode.git/clone" did not exist on "de60a3fb93957dce6b242299b5d163f02ef7f383"

23 Feb, 2021 6 commits
- optional residual · 72e40c74
  Rick Ho authored Feb 23, 2021
  
  72e40c74
- fix tensor initialization bug · e86dea53
  Rick Ho authored Feb 23, 2021
  
  e86dea53
- optional layer-norm in transformer mlp module · 9e67148c
  Rick Ho authored Feb 23, 2021
  
  9e67148c
- megatron support for specific expert parallelism · b97483a4
  Rick Ho authored Feb 23, 2021
  
  b97483a4
- use nn.modulelist to wrap up customized experts · 8f67b530
  Rick Ho authored Feb 23, 2021
  
  8f67b530
- Fix input grad in mp group · 704092b1
  Sengxian authored Feb 23, 2021
  
  704092b1
22 Feb, 2021 5 commits
- fix unpack bug · 092c8d67
  Sengxian authored Feb 22, 2021
  
  092c8d67
- Fix conflict · 05728524
  Sengxian authored Feb 22, 2021
  
  05728524
- customized top_k and add test for localddp · f39f411a
  Sengxian authored Feb 22, 2021
  
  f39f411a
- add inspection of input.grad in tests · 5cb4e63c
  Rick Ho authored Feb 22, 2021
  
  5cb4e63c
- customized dp-comm and hidden-hidden-size · 27c89b5a
  Rick Ho authored Feb 22, 2021
  
  27c89b5a
21 Feb, 2021 9 commits
- Merge pull request #5 from laekov/bias · 87dad9d5
  Rick Ho authored Feb 21, 2021
```
merge Bias
```
  87dad9d5
- Merge remote-tracking branch 'origin/master' into bias · 01464726
  Jiezhong Qiu authored Feb 21, 2021
  
  01464726
- follow pylint to remove lambda · 3d8e8a43
  Jiezhong Qiu authored Feb 21, 2021
  
  3d8e8a43
- test bias grad · e36c3c4c
  Jiezhong Qiu authored Feb 21, 2021
  
  e36c3c4c
- new test structure according to new expert_fn structure · 6acc3e41
  Rick Ho authored Feb 21, 2021
  
  6acc3e41
- change expert_fn structure · c0a3a425
  Rick Ho authored Feb 21, 2021
  
  c0a3a425
- test bias, dp still not passed · a88d1124
  Jiezhong Qiu authored Feb 21, 2021
  
  a88d1124
- update nccl installation method and write readme · ed9277f9
  Rick Ho authored Feb 21, 2021
  
  ed9277f9
- add and initialize bias term in FMoELinear · 3c24222c
  Jiezhong Qiu authored Feb 21, 2021
  
  3c24222c
20 Feb, 2021 7 commits
- Merge pull request #4 from laekov/init_expert · 406955e7
  Rick Ho authored Feb 20, 2021
```
merge init expert
```
  406955e7
- fix pylint issues · 269f3fd4
  Jiezhong Qiu authored Feb 20, 2021
  
  269f3fd4
- remove nccl from lib list · 1a6073b5
  Jiezhong Qiu authored Feb 20, 2021
  
  1a6073b5
- Merge remote-tracking branch 'origin/master' into init_expert · fa6872cd
  Jiezhong Qiu authored Feb 20, 2021
  
  fa6872cd
- add nccl to lib list · 02ead75d
  Rick Ho authored Feb 20, 2021
  
  02ead75d
- fix import error and dtype/device problem in MoE · da4a8e5e
  Jiezhong Qiu authored Feb 20, 2021
  
  da4a8e5e
- init FMoELinear param using mp_ank and numpy rng · 5f5ccd47
  Jiezhong Qiu authored Feb 20, 2021
  
  5f5ccd47
19 Feb, 2021 1 commit
- broadcast distributed parameters · da11cb76
  Rick Ho authored Feb 19, 2021
  
  da11cb76
18 Feb, 2021 2 commits
- update setup.py and requirements.txt · afd43f51
  Rick Ho authored Feb 18, 2021
  
  afd43f51
- add license · 709ac745
  Rick Ho authored Feb 18, 2021
  
  709ac745
09 Feb, 2021 1 commit
- Skip test when there are no enough gpu · 7466180f
  Sengxian authored Feb 09, 2021
  
  7466180f
08 Feb, 2021 9 commits
- Add model parallel test for FMoE with ddp · dad99892
  Sengxian authored Feb 08, 2021
  
  dad99892
- Seperate ddp test from test_numerical · 06e75b3a
  Sengxian authored Feb 08, 2021
  
  06e75b3a
- Merge pull request #3 from laekov/laekov/benchmarks · 1a72a0cb
  Rick Ho authored Feb 08, 2021
```
separate benchmark with tests
```
  1a72a0cb
- adapt benchmark to new moe module · 101b847c
  Rick Ho authored Feb 08, 2021
  
  101b847c
- merge new tests · 8dac1a52
  Rick Ho authored Feb 08, 2021
  
  8dac1a52
- Add DataParallel test for FMoE · 34477955
  Sengxian authored Feb 08, 2021
  
  34477955
- Fix top_k=3 testcases · 40841453
  Sengxian authored Feb 08, 2021
  
  40841453
- Merge pull request #2 from test · baf2b118
  Rick Ho authored Feb 08, 2021
```
Add test for FMoE layer
```
  baf2b118
- Fix bug in DDP test · 184b0404
  Sengxian authored Feb 08, 2021
  
  184b0404