Commits · b56c8043fa8a2a8704fcc28e659e46519e92c091 · OpenDAS / FastMoE

24 Feb, 2021 1 commit
- add final dropout before residual · 61e4533f
  Jiezhong Qiu authored Feb 24, 2021
  
  61e4533f
23 Feb, 2021 1 commit
- optional layer-norm in transformer mlp module · 9e67148c
  Rick Ho authored Feb 23, 2021
  
  9e67148c
22 Feb, 2021 1 commit
- customized dp-comm and hidden-hidden-size · 27c89b5a
  Rick Ho authored Feb 22, 2021
  
  27c89b5a
21 Feb, 2021 2 commits
- change expert_fn structure · c0a3a425
  Rick Ho authored Feb 21, 2021
  
  c0a3a425
- test bias, dp still not passed · a88d1124
  Jiezhong Qiu authored Feb 21, 2021
  
  a88d1124
20 Feb, 2021 2 commits
- fix pylint issues · 269f3fd4
  Jiezhong Qiu authored Feb 20, 2021
  
  269f3fd4
- init FMoELinear param using mp_ank and numpy rng · 5f5ccd47
  Jiezhong Qiu authored Feb 20, 2021
  
  5f5ccd47
08 Feb, 2021 1 commit
- newer test with top-k bug fixed · c5556037
  Rick Ho authored Feb 08, 2021
  
  c5556037
07 Feb, 2021 1 commit
- support arbitrary module as expert · b3380ec2
  Rick Ho authored Feb 07, 2021
  
  b3380ec2