Commits · 56dd6dd4e9895841840f92ad93231672bd45449e · OpenDAS / FastMoE

10 Nov, 2021 1 commit
- Update proj_adaptive_softmax.py · 56dd6dd4
  Zilin Zhu authored Nov 10, 2021
  
  56dd6dd4
09 Nov, 2021 1 commit
- update setup.py to be compatible with torch v1.10.0 (#88) · b294221d
  Rick Ho authored Nov 10, 2021
  
  b294221d
08 Nov, 2021 3 commits
- Merge pull request #86 from laekov/v0.3.0-rc · acf8bec8
  Rick Ho authored Nov 08, 2021
```
v0.3.0 Release
```
  acf8bec8
- fix typo · a461be6c
  Rick Ho authored Nov 08, 2021
  
  a461be6c
- update release note · 254cdbbc
  Rick Ho authored Nov 08, 2021
  
  254cdbbc
26 Oct, 2021 7 commits
- Merge pull request #84 from laekov/swipe · 3397bc19
  Rick Ho authored Oct 26, 2021
```
SWIPE balance strategy
```
  3397bc19
- fix swipe eval · 206f267e
  Rick Ho authored Oct 26, 2021
  
  206f267e
- fix megatron adapter for swipe · 57bdfe88
  Rick Ho authored Oct 26, 2021
  
  57bdfe88
- swipe pass test · 8a56481b
  Rick Ho authored Oct 26, 2021
  
  8a56481b
- swipe test not passing · c5cbd64b
  Rick Ho authored Oct 26, 2021
  
  c5cbd64b
- fmoe python class update to support comm group in gate · fe5a9cda
  Rick Ho authored Oct 26, 2021
  
  fe5a9cda
- cpu swipe · 8052b3c0
  Rick Ho authored Oct 26, 2021
  
  8052b3c0
25 Oct, 2021 4 commits
- Merge pull request #81 from laekov/realmp · 4a9ef7fd
  Rick Ho authored Oct 25, 2021
```
Model parallelism of single experts
```
  4a9ef7fd
- grammar fix · d30585dc
  Rick Ho authored Oct 25, 2021
  
  d30585dc
- update version number · 5a6e521e
  Rick Ho authored Oct 25, 2021
  
  5a6e521e
- bug fix for megatron patch · a8c5a3cb
  Rick Ho authored Oct 25, 2021
  
  a8c5a3cb
12 Oct, 2021 1 commit
- update calculation for megatron hhs · ae10e942
  Rick Ho authored Oct 12, 2021
  
  ae10e942
10 Oct, 2021 4 commits
- fix zero test · 07a5d8ac
  Rick Ho authored Oct 10, 2021
  
  07a5d8ac
- tide up megatron compatible layer · 5f8ba136
  Rick Ho authored Oct 10, 2021
  
  5f8ba136
- update unit test · 33fc3aca
  Rick Ho authored Oct 10, 2021
  
  33fc3aca
- tide up fmoe python code · d9dba929
  Rick Ho authored Oct 10, 2021
  
  d9dba929
30 Sep, 2021 7 commits
- Merge pull request #78 from hclearner/fastmoe-rocm4.0.1 · 368c8e41
  Rick Ho authored Sep 30, 2021
```
add fastmoe support rocm4.0.1
```
  368c8e41
- Update setup.py · 49e81f33
  hclearner authored Sep 30, 2021
  
  49e81f33
- Update helper_cuda.h · 8ebdc789
  hclearner authored Sep 30, 2021
  
  8ebdc789
- Update cublas_wrapper.h · 9d0f67d6
  hclearner authored Sep 30, 2021
  
  9d0f67d6
- Update cublas_wrapper.h · 43eabe75
  hclearner authored Sep 30, 2021
  
  43eabe75
- Update local_exchange.cuh · 976ef2b8
  hclearner authored Sep 30, 2021
  
  976ef2b8
- add fastmoe support rocm4.0.1 · 2a77b772
  huchen1 authored Sep 30, 2021
  
  2a77b772
23 Aug, 2021 2 commits
- Merge pull request #70 from laekov/v0.2.1-rc1 · d2392de2
  Rick Ho authored Aug 23, 2021
```
Release v0.2.1
```
  d2392de2
- checkout to 0.2.1 · a0c4ed76
  Rick Ho authored Aug 23, 2021
  
  a0c4ed76
18 Aug, 2021 2 commits
- Merge pull request #68 from serendipityCoding/fix-transformer-typo · 8bd400d0
  Rick Ho authored Aug 18, 2021
```
Fix typo in transformer example
```
  8bd400d0
- fix typo in transformer example · 0ac24914
  serendipityCoding authored Aug 18, 2021
  
  0ac24914
04 Aug, 2021 1 commit
- Merge pull request #66 from laekov/slurm-benchmarks · 0ec96584
  Rick Ho authored Aug 04, 2021
```
Update benchmark according to update of gates
```
  0ec96584
02 Aug, 2021 3 commits
- Merge pull request #65 from laekov/megatron-grad-clip · f93ad285
  Rick Ho authored Aug 02, 2021
```
Grad clip patch for megatron v2.2
```
  f93ad285
- fix mathametical bug · b6d280c5
  Rick Ho authored Aug 02, 2021
  
  b6d280c5
- Update bug_report.md · 0fb332dd
  Rick Ho authored Aug 02, 2021
  
  0fb332dd
30 Jul, 2021 2 commits
- update benchmark accordingly · 98584dd0
  Rick Ho authored Jul 30, 2021
  
  98584dd0
- add patch for v2.2 · bc4d35b8
  Rick Ho authored Jul 30, 2021
  
  bc4d35b8
20 Jul, 2021 2 commits
- Merge pull request #64 from laekov/fp16_bal_loss · bba5f289
  Rick Ho authored Jul 20, 2021
```
fix fp16 training with balance loss
```
  bba5f289
- fix fp16 training with balance loss · ebefe2b1
  Jiezhong Qiu authored Jul 20, 2021
```
https://github.com/laekov/fastmoe/issues/63
```
  ebefe2b1