Commits · d2392de2867bc736b62d7d8d3bb2559a964dd520 · OpenDAS / FastMoE

07 Jul, 2021 1 commit
- add moe_comm · 18a4395c
  Rick Ho authored Jul 08, 2021
  
  18a4395c
23 May, 2021 1 commit
- mask and experts list · 28ba2d28
  Colin authored May 23, 2021
  
  28ba2d28
22 Mar, 2021 2 commits
- fix pylint · c844413b
  Jiezhong Qiu authored Mar 16, 2021
  
  c844413b
- Add gshard and noisy gate balance strategy · 89d6c794
  Sengxian authored Mar 22, 2021
  
  89d6c794
16 Mar, 2021 1 commit
- fix pylint · a54546a4
  Jiezhong Qiu authored Mar 16, 2021
  
  a54546a4
13 Mar, 2021 1 commit
- fix lint · 3b82e379
  Rick Ho authored Mar 13, 2021
  
  3b82e379
09 Mar, 2021 1 commit
- Format using black · 4d48209d
  Sengxian authored Mar 09, 2021
  
  4d48209d
25 Feb, 2021 1 commit
- simplify TransformerMLP · fe2009b1
  Jiezhong Qiu authored Feb 26, 2021
```
move away layernorm/residual/dropout
```
  fe2009b1
24 Feb, 2021 1 commit
- add final dropout before residual · 61e4533f
  Jiezhong Qiu authored Feb 24, 2021
  
  61e4533f
23 Feb, 2021 2 commits
- optional residual · 72e40c74
  Rick Ho authored Feb 23, 2021
  
  72e40c74
- optional layer-norm in transformer mlp module · 9e67148c
  Rick Ho authored Feb 23, 2021
  
  9e67148c
22 Feb, 2021 1 commit
- customized dp-comm and hidden-hidden-size · 27c89b5a
  Rick Ho authored Feb 22, 2021
  
  27c89b5a
21 Feb, 2021 2 commits
- change expert_fn structure · c0a3a425
  Rick Ho authored Feb 21, 2021
  
  c0a3a425
- test bias, dp still not passed · a88d1124
  Jiezhong Qiu authored Feb 21, 2021
  
  a88d1124
20 Feb, 2021 2 commits
- fix pylint issues · 269f3fd4
  Jiezhong Qiu authored Feb 20, 2021
  
  269f3fd4
- init FMoELinear param using mp_ank and numpy rng · 5f5ccd47
  Jiezhong Qiu authored Feb 20, 2021
  
  5f5ccd47
08 Feb, 2021 1 commit
- newer test with top-k bug fixed · c5556037
  Rick Ho authored Feb 08, 2021
  
  c5556037
07 Feb, 2021 1 commit
- support arbitrary module as expert · b3380ec2
  Rick Ho authored Feb 07, 2021
  
  b3380ec2