Commits · e36c3c4ce1fb6311c2d649316ddf7a14e2684581 · OpenDAS / FastMoE

21 Feb, 2021 5 commits
- test bias grad · e36c3c4c
  Jiezhong Qiu authored Feb 21, 2021
  
  e36c3c4c
- new test structure according to new expert_fn structure · 6acc3e41
  Rick Ho authored Feb 21, 2021
  
  6acc3e41
- change expert_fn structure · c0a3a425
  Rick Ho authored Feb 21, 2021
  
  c0a3a425
- test bias, dp still not passed · a88d1124
  Jiezhong Qiu authored Feb 21, 2021
  
  a88d1124
- add and initialize bias term in FMoELinear · 3c24222c
  Jiezhong Qiu authored Feb 21, 2021
  
  3c24222c
20 Feb, 2021 7 commits
- Merge pull request #4 from laekov/init_expert · 406955e7
  Rick Ho authored Feb 20, 2021
```
merge init expert
```
  406955e7
- fix pylint issues · 269f3fd4
  Jiezhong Qiu authored Feb 20, 2021
  
  269f3fd4
- remove nccl from lib list · 1a6073b5
  Jiezhong Qiu authored Feb 20, 2021
  
  1a6073b5
- Merge remote-tracking branch 'origin/master' into init_expert · fa6872cd
  Jiezhong Qiu authored Feb 20, 2021
  
  fa6872cd
- add nccl to lib list · 02ead75d
  Rick Ho authored Feb 20, 2021
  
  02ead75d
- fix import error and dtype/device problem in MoE · da4a8e5e
  Jiezhong Qiu authored Feb 20, 2021
  
  da4a8e5e
- init FMoELinear param using mp_ank and numpy rng · 5f5ccd47
  Jiezhong Qiu authored Feb 20, 2021
  
  5f5ccd47
19 Feb, 2021 1 commit
- broadcast distributed parameters · da11cb76
  Rick Ho authored Feb 19, 2021
  
  da11cb76
18 Feb, 2021 2 commits
- update setup.py and requirements.txt · afd43f51
  Rick Ho authored Feb 18, 2021
  
  afd43f51
- add license · 709ac745
  Rick Ho authored Feb 18, 2021
  
  709ac745
09 Feb, 2021 1 commit
- Skip test when there are no enough gpu · 7466180f
  Sengxian authored Feb 09, 2021
  
  7466180f
08 Feb, 2021 12 commits
- Add model parallel test for FMoE with ddp · dad99892
  Sengxian authored Feb 08, 2021
  
  dad99892
- Seperate ddp test from test_numerical · 06e75b3a
  Sengxian authored Feb 08, 2021
  
  06e75b3a
- Merge pull request #3 from laekov/laekov/benchmarks · 1a72a0cb
  Rick Ho authored Feb 08, 2021
```
separate benchmark with tests
```
  1a72a0cb
- adapt benchmark to new moe module · 101b847c
  Rick Ho authored Feb 08, 2021
  
  101b847c
- merge new tests · 8dac1a52
  Rick Ho authored Feb 08, 2021
  
  8dac1a52
- Add DataParallel test for FMoE · 34477955
  Sengxian authored Feb 08, 2021
  
  34477955
- Fix top_k=3 testcases · 40841453
  Sengxian authored Feb 08, 2021
  
  40841453
- Merge pull request #2 from test · baf2b118
  Rick Ho authored Feb 08, 2021
```
Add test for FMoE layer
```
  baf2b118
- Fix bug in DDP test · 184b0404
  Sengxian authored Feb 08, 2021
  
  184b0404
- Add test for arbitrary expert in FMoE · 5ead59db
  Sengxian authored Feb 08, 2021
  
  5ead59db
- a stronger benchmark · d2678111
  Rick Ho authored Feb 08, 2021
  
  d2678111
- newer test with top-k bug fixed · c5556037
  Rick Ho authored Feb 08, 2021
  
  c5556037
07 Feb, 2021 6 commits
- Add test for FMoE · fc78d5c3
  Sengxian authored Feb 07, 2021
  
  fc78d5c3
- strong compatibility to different python and pytorch versions · 103343ca
  Rick Ho authored Feb 07, 2021
  
  103343ca
- fix typo · d6169eb1
  Rick Ho authored Feb 07, 2021
  
  d6169eb1
- update the mem-transformer example · fb3e3c29
  Rick Ho authored Feb 07, 2021
  
  fb3e3c29
- support arbitrary module as expert · b3380ec2
  Rick Ho authored Feb 07, 2021
  
  b3380ec2
- separate gates file · 8328c794
  Rick Ho authored Feb 07, 2021
  
  8328c794
05 Feb, 2021 6 commits
- fit fmoe in transformer-xl · 9c92be55
  Rick Ho authored Feb 05, 2021
  
  9c92be55
- do not require comm in non-nccl environment · 5e9bb2e9
  Rick Ho authored Feb 05, 2021
  
  5e9bb2e9
- readme in transformer-xl example · 8f1f2ca5
  Rick Ho authored Feb 05, 2021
  
  8f1f2ca5
- update instructions for megatron · 59b27103
  Rick Ho authored Feb 05, 2021
  
  59b27103
- sync in the whole world instead of mp world in megatron · d6e7a429
  Rick Ho authored Feb 05, 2021
  
  d6e7a429
- pass pylint · f2040d9f
  Rick Ho authored Feb 05, 2021
  
  f2040d9f