Commits · 527a8cc9305bc89e76d98d18908f7bf05e01c310 · OpenDAS / FastMoE

25 Feb, 2021 1 commit
- simplify TransformerMLP · fe2009b1
  Jiezhong Qiu authored Feb 26, 2021
```
move away layernorm/residual/dropout
```
  fe2009b1
24 Feb, 2021 1 commit
- add final dropout before residual · 61e4533f
  Jiezhong Qiu authored Feb 24, 2021
  
  61e4533f
23 Feb, 2021 2 commits
- optional residual · 72e40c74
  Rick Ho authored Feb 23, 2021
  
  72e40c74
- optional layer-norm in transformer mlp module · 9e67148c
  Rick Ho authored Feb 23, 2021
  
  9e67148c
22 Feb, 2021 1 commit
- customized dp-comm and hidden-hidden-size · 27c89b5a
  Rick Ho authored Feb 22, 2021
  
  27c89b5a
21 Feb, 2021 2 commits
- change expert_fn structure · c0a3a425
  Rick Ho authored Feb 21, 2021
  
  c0a3a425
- test bias, dp still not passed · a88d1124
  Jiezhong Qiu authored Feb 21, 2021
  
  a88d1124
20 Feb, 2021 2 commits
- fix pylint issues · 269f3fd4
  Jiezhong Qiu authored Feb 20, 2021
  
  269f3fd4
- init FMoELinear param using mp_ank and numpy rng · 5f5ccd47
  Jiezhong Qiu authored Feb 20, 2021
  
  5f5ccd47
08 Feb, 2021 1 commit
- newer test with top-k bug fixed · c5556037
  Rick Ho authored Feb 08, 2021
  
  c5556037
07 Feb, 2021 1 commit
- support arbitrary module as expert · b3380ec2
  Rick Ho authored Feb 07, 2021
  
  b3380ec2