Commits · 8052b3c0eaff8bde0b427ffcd8f514cd083e1411 · OpenDAS / FastMoE

26 Oct, 2021 1 commit
- cpu swipe · 8052b3c0
  Rick Ho authored Oct 26, 2021
  
  8052b3c0
25 Oct, 2021 4 commits
- Merge pull request #81 from laekov/realmp · 4a9ef7fd
  Rick Ho authored Oct 25, 2021
```
Model parallelism of single experts
```
  4a9ef7fd
- grammar fix · d30585dc
  Rick Ho authored Oct 25, 2021
  
  d30585dc
- update version number · 5a6e521e
  Rick Ho authored Oct 25, 2021
  
  5a6e521e
- bug fix for megatron patch · a8c5a3cb
  Rick Ho authored Oct 25, 2021
  
  a8c5a3cb
12 Oct, 2021 1 commit
- update calculation for megatron hhs · ae10e942
  Rick Ho authored Oct 12, 2021
  
  ae10e942
10 Oct, 2021 4 commits
- fix zero test · 07a5d8ac
  Rick Ho authored Oct 10, 2021
  
  07a5d8ac
- tide up megatron compatible layer · 5f8ba136
  Rick Ho authored Oct 10, 2021
  
  5f8ba136
- update unit test · 33fc3aca
  Rick Ho authored Oct 10, 2021
  
  33fc3aca
- tide up fmoe python code · d9dba929
  Rick Ho authored Oct 10, 2021
  
  d9dba929
30 Sep, 2021 7 commits
- Merge pull request #78 from hclearner/fastmoe-rocm4.0.1 · 368c8e41
  Rick Ho authored Sep 30, 2021
```
add fastmoe support rocm4.0.1
```
  368c8e41
- Update setup.py · 49e81f33
  hclearner authored Sep 30, 2021
  
  49e81f33
- Update helper_cuda.h · 8ebdc789
  hclearner authored Sep 30, 2021
  
  8ebdc789
- Update cublas_wrapper.h · 9d0f67d6
  hclearner authored Sep 30, 2021
  
  9d0f67d6
- Update cublas_wrapper.h · 43eabe75
  hclearner authored Sep 30, 2021
  
  43eabe75
- Update local_exchange.cuh · 976ef2b8
  hclearner authored Sep 30, 2021
  
  976ef2b8
- add fastmoe support rocm4.0.1 · 2a77b772
  huchen1 authored Sep 30, 2021
  
  2a77b772
23 Aug, 2021 2 commits
- Merge pull request #70 from laekov/v0.2.1-rc1 · d2392de2
  Rick Ho authored Aug 23, 2021
```
Release v0.2.1
```
  d2392de2
- checkout to 0.2.1 · a0c4ed76
  Rick Ho authored Aug 23, 2021
  
  a0c4ed76
18 Aug, 2021 2 commits
- Merge pull request #68 from serendipityCoding/fix-transformer-typo · 8bd400d0
  Rick Ho authored Aug 18, 2021
```
Fix typo in transformer example
```
  8bd400d0
- fix typo in transformer example · 0ac24914
  serendipityCoding authored Aug 18, 2021
  
  0ac24914
04 Aug, 2021 1 commit
- Merge pull request #66 from laekov/slurm-benchmarks · 0ec96584
  Rick Ho authored Aug 04, 2021
```
Update benchmark according to update of gates
```
  0ec96584
02 Aug, 2021 3 commits
- Merge pull request #65 from laekov/megatron-grad-clip · f93ad285
  Rick Ho authored Aug 02, 2021
```
Grad clip patch for megatron v2.2
```
  f93ad285
- fix mathametical bug · b6d280c5
  Rick Ho authored Aug 02, 2021
  
  b6d280c5
- Update bug_report.md · 0fb332dd
  Rick Ho authored Aug 02, 2021
  
  0fb332dd
30 Jul, 2021 2 commits
- update benchmark accordingly · 98584dd0
  Rick Ho authored Jul 30, 2021
  
  98584dd0
- add patch for v2.2 · bc4d35b8
  Rick Ho authored Jul 30, 2021
  
  bc4d35b8
20 Jul, 2021 2 commits
- Merge pull request #64 from laekov/fp16_bal_loss · bba5f289
  Rick Ho authored Jul 20, 2021
```
fix fp16 training with balance loss
```
  bba5f289
- fix fp16 training with balance loss · ebefe2b1
  Jiezhong Qiu authored Jul 20, 2021
```
https://github.com/laekov/fastmoe/issues/63
```
  ebefe2b1
08 Jul, 2021 4 commits
- Merge pull request #60 from laekov/remove-comm · 537679a8
  Rick Ho authored Jul 08, 2021
```
Remove unnecessary dependencies on comm
```
  537679a8
- remove unnecessary dependencies on comm · c8483d42
  Rick Ho authored Jul 08, 2021
  
  c8483d42
- Merge pull request #59 from laekov/cope-with-pipeline · 50a9aa94
  Rick Ho authored Jul 08, 2021
```
Use moe_group instead of world for MoE
```
  50a9aa94
- resolve loss reduction with customized gates · 59913cca
  Rick Ho authored Jul 08, 2021
  
  59913cca
07 Jul, 2021 1 commit
- add moe_comm · 18a4395c
  Rick Ho authored Jul 08, 2021
  
  18a4395c
30 Jun, 2021 2 commits
- Merge pull request #57 from ymjiang/patch-1 · 55f8ca7d
  Rick Ho authored Jun 30, 2021
```
Fix typo in readme
```
  55f8ca7d
- Fix typo in readme · a4119185
  Yimin Jiang authored Jun 30, 2021
  
  a4119185
29 Jun, 2021 1 commit
- Merge pull request #56 from laekov/older-pytorch-compatibility · 11369d67
  Jiezhong Qiu authored Jun 29, 2021
```
Fix pytorch compatibility issue !55
```
  11369d67
28 Jun, 2021 2 commits
- acro bug fix · 9170835c
  Rich Ho authored Jun 28, 2021
  
  9170835c
- fix pytorch compatibility issue !55 · 17e2a5c5
  Rick Ho authored Jun 28, 2021
  
  17e2a5c5
18 Jun, 2021 1 commit
- Added default weight initializations to FMoELinear and NoisyGate (#52) · 7d41fe88
  Tiago Antunes authored Jun 18, 2021
```
* Added default weight initializations to FMoELinear and NoisyGate

* Following torch's naming convention
```
  7d41fe88