Commits · 07a5d8ac7b0b138e20727cc4b3acffbe9b8d8445 · OpenDAS / FastMoE

10 Oct, 2021 1 commit
- tide up megatron compatible layer · 5f8ba136
  Rick Ho authored Oct 10, 2021
  
  5f8ba136
20 Jul, 2021 1 commit
- fix fp16 training with balance loss · ebefe2b1
  Jiezhong Qiu authored Jul 20, 2021
```
https://github.com/laekov/fastmoe/issues/63
```
  ebefe2b1
08 Jul, 2021 1 commit
- resolve loss reduction with customized gates · 59913cca
  Rick Ho authored Jul 08, 2021
  
  59913cca
07 Jul, 2021 1 commit
- add moe_comm · 18a4395c
  Rick Ho authored Jul 08, 2021
  
  18a4395c
17 Jun, 2021 2 commits
- fix concat shape · a12ad553
  Rick Ho authored Jun 17, 2021
  
  a12ad553
- use cat instead of creating new tensor · 913d7127
  Rick Ho authored Jun 17, 2021
  
  913d7127
31 May, 2021 2 commits
- Add hidden hidden size args · f9ce8e09
  Sengxian authored May 31, 2021
  
  f9ce8e09
- Adapt balance loss for new gate interface & update patch · 4eec9807
  Sengxian authored May 31, 2021
  
  4eec9807
22 Mar, 2021 2 commits
- add balance to init · 2bceb76b
  Rick Ho authored Mar 22, 2021
  
  2bceb76b
- initial split · b690f245
  Rick Ho authored Mar 22, 2021
  
  b690f245