Commits · 450549a4f52fd2d116cfa2652609deceda41dc8f · OpenDAS / FastMoE

26 Apr, 2021 2 commits
- fix zero gate and constant gate · 450549a4
  Rick Ho authored Apr 26, 2021
  
  450549a4
- add gshard and switch gate with loss · 4c90e6e8
  Rick Ho authored Apr 26, 2021
  
  4c90e6e8
23 Mar, 2021 1 commit
- fix tests after updating megatron · 6868ed2a
  Rick Ho authored Mar 24, 2021
  
  6868ed2a
22 Mar, 2021 1 commit
- Add gshard and noisy gate balance strategy · 89d6c794
  Sengxian authored Mar 22, 2021
  
  89d6c794
13 Mar, 2021 1 commit
- manually broadcast nccl unique id · 43af1522
  Rick Ho authored Mar 13, 2021
  
  43af1522
09 Mar, 2021 1 commit
- Format using black · 4d48209d
  Sengxian authored Mar 09, 2021
  
  4d48209d
08 Mar, 2021 1 commit
- fix numerical issue in tests and typo · 527a8cc9
  Rick Ho authored Mar 08, 2021
  
  527a8cc9
22 Feb, 2021 3 commits
- fix unpack bug · 092c8d67
  Sengxian authored Feb 22, 2021
  
  092c8d67
- customized top_k and add test for localddp · f39f411a
  Sengxian authored Feb 22, 2021
  
  f39f411a
- add inspection of input.grad in tests · 5cb4e63c
  Rick Ho authored Feb 22, 2021
  
  5cb4e63c
21 Feb, 2021 3 commits
- test bias grad · e36c3c4c
  Jiezhong Qiu authored Feb 21, 2021
  
  e36c3c4c
- new test structure according to new expert_fn structure · 6acc3e41
  Rick Ho authored Feb 21, 2021
  
  6acc3e41
- test bias, dp still not passed · a88d1124
  Jiezhong Qiu authored Feb 21, 2021
  
  a88d1124
09 Feb, 2021 1 commit
- Skip test when there are no enough gpu · 7466180f
  Sengxian authored Feb 09, 2021
  
  7466180f
08 Feb, 2021 9 commits
- Add model parallel test for FMoE with ddp · dad99892
  Sengxian authored Feb 08, 2021
  
  dad99892
- Seperate ddp test from test_numerical · 06e75b3a
  Sengxian authored Feb 08, 2021
  
  06e75b3a
- adapt benchmark to new moe module · 101b847c
  Rick Ho authored Feb 08, 2021
  
  101b847c
- Add DataParallel test for FMoE · 34477955
  Sengxian authored Feb 08, 2021
  
  34477955
- Fix top_k=3 testcases · 40841453
  Sengxian authored Feb 08, 2021
  
  40841453
- Fix bug in DDP test · 184b0404
  Sengxian authored Feb 08, 2021
  
  184b0404
- Add test for arbitrary expert in FMoE · 5ead59db
  Sengxian authored Feb 08, 2021
  
  5ead59db
- a stronger benchmark · d2678111
  Rick Ho authored Feb 08, 2021
  
  d2678111
- newer test with top-k bug fixed · c5556037
  Rick Ho authored Feb 08, 2021
  
  c5556037
07 Feb, 2021 2 commits
- Add test for FMoE · fc78d5c3
  Sengxian authored Feb 07, 2021
  
  fc78d5c3
- strong compatibility to different python and pytorch versions · 103343ca
  Rick Ho authored Feb 07, 2021
  
  103343ca
02 Feb, 2021 1 commit
- remove debug output and todo for replicated mp input · a8ecd3d7
  Rick Ho authored Feb 02, 2021
  
  a8ecd3d7
01 Feb, 2021 3 commits
- fix typo in test_all.sh · f9bec836
  Rick Ho authored Feb 01, 2021
  
  f9bec836
- split more tests · 01d9b418
  Rick Ho authored Feb 01, 2021
  
  01d9b418
- complete test for reconstruction · 22e1eb45
  Rick Ho authored Feb 01, 2021
  
  22e1eb45
29 Jan, 2021 1 commit
- scatter gather kernel support for non-equal shapes and fix tests · 72e9bc9e
  Rick Ho authored Jan 29, 2021
  
  72e9bc9e
28 Jan, 2021 2 commits
- fix scatter/gather bug to make it correct · 952e3135
  Rick Ho authored Jan 28, 2021
  
  952e3135
- make test run on nccl version, but fails in correctness · 2d250fbf
  Rick Ho authored Jan 28, 2021
  
  2d250fbf
25 Jan, 2021 3 commits
- basic megatron support frame · d0f07ff7
  Rick Ho authored Jan 25, 2021
  
  d0f07ff7
- pass test · c5cfd5fb
  Rick Ho authored Jan 25, 2021
  
  c5cfd5fb
- update fmoe files · ed69591a
  Rick Ho authored Jan 25, 2021
  
  ed69591a