Commits · 21a331fbea2f198b68b3bc2b8304d0a092a940c2 · OpenDAS / FastMoE

01 Jun, 2022 1 commit
- add a hint for DGDP synchronization · 21a331fb
  Rick Ho authored Jun 01, 2022
  
  21a331fb
26 May, 2022 1 commit
- fix schedule with older autograd · db247e85
  Rick Ho authored May 26, 2022
  
  db247e85
30 Apr, 2022 4 commits
- add condition of skipping pop params · 4682c1d0
  zms1999 authored Apr 30, 2022
  
  4682c1d0
- fix type mismatch · 9f7cbd15
  zms1999 authored Apr 30, 2022
  
  9f7cbd15
- fix shape mismatch in set_grad · 0edd811d
  zms1999 authored Apr 30, 2022
  
  0edd811d
- fix typo in shadow policy · d90ff389
  zms1999 authored Apr 30, 2022
  
  d90ff389
02 Apr, 2022 1 commit
- some comments on code and test · 1b8fef31
  Rick Ho authored Apr 02, 2022
  
  1b8fef31
01 Apr, 2022 1 commit
- faster policies · 91a5e794
  Rick Ho authored Apr 01, 2022
  
  91a5e794
31 Mar, 2022 1 commit
- expert shadow backward with test · 794dd0e6
  Rick Ho authored Mar 31, 2022
  
  794dd0e6
30 Mar, 2022 2 commits
- forward tested · b5b72d41
  Rick Ho authored Mar 30, 2022
  
  b5b72d41
- forward code in smart schedule · 771dc62d
  Rick Ho authored Mar 30, 2022
  
  771dc62d
29 Mar, 2022 3 commits
- fix backward grad weight bug · 6c68b56b
  Rick Ho authored Mar 30, 2022
  
  6c68b56b
- backward bugous on grad weight · a807e2a3
  Rick Ho authored Mar 29, 2022
  
  a807e2a3
- forward pass test · 46c3722d
  Rick Ho authored Mar 29, 2022
  
  46c3722d
28 Mar, 2022 3 commits
- faster gate develop · 6e1fcca1
  Rick Ho authored Mar 28, 2022
  
  6e1fcca1
- fix document for megatron · 7bdf58c9
  Rick Ho authored Mar 28, 2022
  
  7bdf58c9
- Fix GShardGate random_routing · cfd30236
  zms1999 authored Mar 28, 2022
  
  cfd30236
29 Nov, 2021 1 commit
- fix naive gate output score · a6ca7f62
  Rick Ho authored Nov 29, 2021
  
  a6ca7f62
23 Nov, 2021 4 commits
- remove non_moe_inp · 0abea7b2
  Jiezhong Qiu authored Nov 23, 2021
  
  0abea7b2
- check moe input/output batch sizes are the same · 1c555604
  Jiezhong Qiu authored Nov 23, 2021
  
  1c555604
- fix and test moe with multiple input/output · 27c8c2f3
  Jiezhong Qiu authored Nov 23, 2021
  
  27c8c2f3
- allow netsd data struct as moe input · 2d81858b
  Jiezhong Qiu authored Nov 23, 2021
  
  2d81858b
08 Nov, 2021 1 commit
- update release note · 254cdbbc
  Rick Ho authored Nov 08, 2021
  
  254cdbbc
26 Oct, 2021 5 commits
- fix swipe eval · 206f267e
  Rick Ho authored Oct 26, 2021
  
  206f267e
- fix megatron adapter for swipe · 57bdfe88
  Rick Ho authored Oct 26, 2021
  
  57bdfe88
- swipe test not passing · c5cbd64b
  Rick Ho authored Oct 26, 2021
  
  c5cbd64b
- fmoe python class update to support comm group in gate · fe5a9cda
  Rick Ho authored Oct 26, 2021
  
  fe5a9cda
- cpu swipe · 8052b3c0
  Rick Ho authored Oct 26, 2021
  
  8052b3c0
25 Oct, 2021 2 commits
- grammar fix · d30585dc
  Rick Ho authored Oct 25, 2021
  
  d30585dc
- bug fix for megatron patch · a8c5a3cb
  Rick Ho authored Oct 25, 2021
  
  a8c5a3cb
12 Oct, 2021 1 commit
- update calculation for megatron hhs · ae10e942
  Rick Ho authored Oct 12, 2021
  
  ae10e942
10 Oct, 2021 2 commits
- tide up megatron compatible layer · 5f8ba136
  Rick Ho authored Oct 10, 2021
  
  5f8ba136
- tide up fmoe python code · d9dba929
  Rick Ho authored Oct 10, 2021
  
  d9dba929
20 Jul, 2021 1 commit
- fix fp16 training with balance loss · ebefe2b1
  Jiezhong Qiu authored Jul 20, 2021
```
https://github.com/laekov/fastmoe/issues/63
```
  ebefe2b1
08 Jul, 2021 2 commits
- remove unnecessary dependencies on comm · c8483d42
  Rick Ho authored Jul 08, 2021
  
  c8483d42
- resolve loss reduction with customized gates · 59913cca
  Rick Ho authored Jul 08, 2021
  
  59913cca
07 Jul, 2021 1 commit
- add moe_comm · 18a4395c
  Rick Ho authored Jul 08, 2021
  
  18a4395c
18 Jun, 2021 1 commit
- Added default weight initializations to FMoELinear and NoisyGate (#52) · 7d41fe88
  Tiago Antunes authored Jun 18, 2021
```
* Added default weight initializations to FMoELinear and NoisyGate

* Following torch's naming convention
```
  7d41fe88
17 Jun, 2021 2 commits
- use single variable for returned value · 318513ae
  Jiezhong Qiu authored Jun 17, 2021
```
the old impl raised error "too many values to unpack (expected 1)"
```
  318513ae
- fix concat shape · a12ad553
  Rick Ho authored Jun 17, 2021
  
  a12ad553