Commits · 89d6c794748cf171c74dd680b5a64fa0f422d220 · OpenDAS / FastMoE

"sgl-kernel/vscode:/vscode.git/clone" did not exist on "d4bf5a8524820d3a232f7fc8e349d5e7d0d2880d"

22 Mar, 2021 1 commit
- Add gshard and noisy gate balance strategy · 89d6c794
  Sengxian authored Mar 22, 2021
  
  89d6c794
13 Mar, 2021 1 commit
- fix lint · 3b82e379
  Rick Ho authored Mar 13, 2021
  
  3b82e379
09 Mar, 2021 1 commit
- Format using black · 4d48209d
  Sengxian authored Mar 09, 2021
  
  4d48209d
25 Feb, 2021 1 commit
- simplify TransformerMLP · fe2009b1
  Jiezhong Qiu authored Feb 26, 2021
```
move away layernorm/residual/dropout
```
  fe2009b1
24 Feb, 2021 1 commit
- add final dropout before residual · 61e4533f
  Jiezhong Qiu authored Feb 24, 2021
  
  61e4533f
23 Feb, 2021 2 commits
- optional residual · 72e40c74
  Rick Ho authored Feb 23, 2021
  
  72e40c74
- optional layer-norm in transformer mlp module · 9e67148c
  Rick Ho authored Feb 23, 2021
  
  9e67148c
22 Feb, 2021 1 commit
- customized dp-comm and hidden-hidden-size · 27c89b5a
  Rick Ho authored Feb 22, 2021
  
  27c89b5a
21 Feb, 2021 2 commits
- change expert_fn structure · c0a3a425
  Rick Ho authored Feb 21, 2021
  
  c0a3a425
- test bias, dp still not passed · a88d1124
  Jiezhong Qiu authored Feb 21, 2021
  
  a88d1124
20 Feb, 2021 2 commits
- fix pylint issues · 269f3fd4
  Jiezhong Qiu authored Feb 20, 2021
  
  269f3fd4
- init FMoELinear param using mp_ank and numpy rng · 5f5ccd47
  Jiezhong Qiu authored Feb 20, 2021
  
  5f5ccd47
08 Feb, 2021 1 commit
- newer test with top-k bug fixed · c5556037
  Rick Ho authored Feb 08, 2021
  
  c5556037
07 Feb, 2021 1 commit
- support arbitrary module as expert · b3380ec2
  Rick Ho authored Feb 07, 2021
  
  b3380ec2