Commits · 4a9ef7fdd7f9b641ec009a49be067483ab4ffcc2 · OpenDAS / FastMoE

25 Oct, 2021 2 commits
- grammar fix · d30585dc
  Rick Ho authored Oct 25, 2021
  
  d30585dc
- bug fix for megatron patch · a8c5a3cb
  Rick Ho authored Oct 25, 2021
  
  a8c5a3cb
12 Oct, 2021 1 commit
- update calculation for megatron hhs · ae10e942
  Rick Ho authored Oct 12, 2021
  
  ae10e942
10 Oct, 2021 1 commit
- tide up megatron compatible layer · 5f8ba136
  Rick Ho authored Oct 10, 2021
  
  5f8ba136
20 Jul, 2021 1 commit
- fix fp16 training with balance loss · ebefe2b1
  Jiezhong Qiu authored Jul 20, 2021
```
https://github.com/laekov/fastmoe/issues/63
```
  ebefe2b1
08 Jul, 2021 1 commit
- resolve loss reduction with customized gates · 59913cca
  Rick Ho authored Jul 08, 2021
  
  59913cca
07 Jul, 2021 1 commit
- add moe_comm · 18a4395c
  Rick Ho authored Jul 08, 2021
  
  18a4395c
17 Jun, 2021 2 commits
- fix concat shape · a12ad553
  Rick Ho authored Jun 17, 2021
  
  a12ad553
- use cat instead of creating new tensor · 913d7127
  Rick Ho authored Jun 17, 2021
  
  913d7127
31 May, 2021 2 commits
- Add hidden hidden size args · f9ce8e09
  Sengxian authored May 31, 2021
  
  f9ce8e09
- Adapt balance loss for new gate interface & update patch · 4eec9807
  Sengxian authored May 31, 2021
  
  4eec9807
22 Mar, 2021 2 commits
- add balance to init · 2bceb76b
  Rick Ho authored Mar 22, 2021
  
  2bceb76b
- initial split · b690f245
  Rick Ho authored Mar 22, 2021
  
  b690f245