Commits · 8328c794d4e22f8db050ea4f8a6e69be3221fe94 · OpenDAS / FastMoE

07 Feb, 2021 1 commit
- separate gates file · 8328c794
  Rick Ho authored Feb 07, 2021
  
  8328c794
05 Feb, 2021 2 commits
- sync in the whole world instead of mp world in megatron · d6e7a429
  Rick Ho authored Feb 05, 2021
  
  d6e7a429
- pass pylint · f2040d9f
  Rick Ho authored Feb 05, 2021
  
  f2040d9f
04 Feb, 2021 2 commits
- fix no grad after all-gather bug · 56c1bd63
  Rick Ho authored Feb 04, 2021
  
  56c1bd63
- use parallel label in gate · d83234b0
  Rick Ho authored Feb 04, 2021
  
  d83234b0
03 Feb, 2021 1 commit
- fmoefy · 6b8d2f2e
  Rick Ho authored Feb 03, 2021
  
  6b8d2f2e
02 Feb, 2021 5 commits
- fix bmm out shape · 4b650671
  Rick Ho authored Feb 02, 2021
  
  4b650671
- fix replica condition and minor optimizations · dc3db673
  Rick Ho authored Feb 02, 2021
  
  dc3db673
- remove debug output and todo for replicated mp input · a8ecd3d7
  Rick Ho authored Feb 02, 2021
  
  a8ecd3d7
- Optimize redundancy communication · 01ae2d72
  Sengxian authored Feb 02, 2021
  
  01ae2d72
- Format using black and add model_parallel_rank · fdbac1df
  Sengxian authored Feb 02, 2021
  
  fdbac1df
01 Feb, 2021 2 commits
- split more tests · 01d9b418
  Rick Ho authored Feb 01, 2021
  
  01d9b418
- complete test for reconstruction · 22e1eb45
  Rick Ho authored Feb 01, 2021
  
  22e1eb45
29 Jan, 2021 3 commits
- fix more bugs to make the layers run in the model · 0fea2991
  Rick Ho authored Jan 29, 2021
  
  0fea2991
- fix python bugs · 6900f1de
  Rick Ho authored Jan 29, 2021
  
  6900f1de
- reconstruct fmoe nn module · 437afda2
  Rick Ho authored Jan 29, 2021
  
  437afda2
27 Jan, 2021 1 commit
- fix expert number · bc8e8181
  Rick Ho authored Jan 27, 2021
  
  bc8e8181
26 Jan, 2021 1 commit
- initial version to run with megatron · 58e949cf
  Rick Ho authored Jan 26, 2021
  
  58e949cf
25 Jan, 2021 3 commits
- basic megatron support frame · d0f07ff7
  Rick Ho authored Jan 25, 2021
  
  d0f07ff7
- pass test · c5cfd5fb
  Rick Ho authored Jan 25, 2021
  
  c5cfd5fb
- update fmoe files · ed69591a
  Rick Ho authored Jan 25, 2021
  
  ed69591a