Commits · 7f15d11dffae91e9dce70bace519739a2a4e23ed · OpenDAS / FastMoE

"vscode:/vscode.git/clone" did not exist on "e118021699f11a0968db2282d69775cfce3497bd"

31 May, 2021 5 commits
- update version number · 7f15d11d
  Rick Ho authored May 31, 2021
  
  7f15d11d
- Merge pull request #41 from laekov/hidden-hidden-size-arg · d205aaeb
  Rick Ho authored May 31, 2021
```
Add hidden hidden size args
```
  d205aaeb
- Add hidden hidden size args · f9ce8e09
  Sengxian authored May 31, 2021
  
  f9ce8e09
- Merge pull request #40 from laekov/new-gate-patch · 4d59a9db
  Rick Ho authored May 31, 2021
```
Adapt balance loss for new gate interface & update patch
```
  4d59a9db
- Adapt balance loss for new gate interface & update patch · 4eec9807
  Sengxian authored May 31, 2021
  
  4eec9807
30 May, 2021 1 commit
- Merge pull request #39 from laekov/gshard-gate-bugfix · c77f676d
  Rick Ho authored May 30, 2021
```
Fix bugs to run megatron with gshard gate
```
  c77f676d
29 May, 2021 1 commit
- fix bugs to run megatron with gshard gate · fa5f45f0
  Rick Ho authored May 29, 2021
  
  fa5f45f0
24 May, 2021 6 commits
- Merge pull request #38 from GODVIX/gates · 7f6463f0
  Rick Ho authored May 24, 2021
```
Update test_gates.py
```
  7f6463f0
- Update test_gates.py · 26cc37cb
  GODVIX authored May 24, 2021
  
  26cc37cb
- Merge pull request #37 from laekov/gshard-random-routing · ddaac5eb
  Rick Ho authored May 24, 2021
```
Add random routing in gshard gate
```
  ddaac5eb
- add random routing in gshard gate · e58e7b3b
  Rich Ho authored May 24, 2021
  
  e58e7b3b
- Merge pull request #36 from Co1lin/master · ba2b7aa9
  Rick Ho authored May 24, 2021
```
mask and experts list
```
  ba2b7aa9
- mask and experts list (#2) · ff7333c7
  Colin authored May 24, 2021
```
- mask some tensors of tokens for fmoe forward
- pass a list of expert classes to specify what experts in what order want to use
```
  ff7333c7
23 May, 2021 1 commit
- mask and experts list · 28ba2d28
  Colin authored May 23, 2021
  
  28ba2d28
21 May, 2021 2 commits
- Merge pull request #31 from laekov/gate · baae8fb9
  Rick Ho authored May 22, 2021
```
Reconstruct gate and add gshard / switch
```
  baae8fb9
- update accoding to comments · 8d14dd29
  Rich Ho authored May 21, 2021
  
  8d14dd29
20 May, 2021 4 commits
- fix zero test · c931c484
  Rich Ho authored May 20, 2021
  
  c931c484
- Merge branch 'master' into laekov/gate · 5680c599
  Rich Ho authored May 20, 2021
  
  5680c599
- Merge pull request #21 from bias_improvement · 3c42c892
  Rick Ho authored May 20, 2021
```
Bias improvement #15
```
  3c42c892
- add data_type in test ddp · 41cfe06c
  Rich Ho authored May 20, 2021
  
  41cfe06c
19 May, 2021 5 commits
- fix scatter bug across gpus · 90c4bccf
  Rich Ho authored May 19, 2021
  
  90c4bccf
- pass test on single gpu · 670d70ed
  Rick Ho authored May 19, 2021
  
  670d70ed
- remove local scatter and gather in cuda · 52e57316
  Rick Ho authored May 19, 2021
  
  52e57316
- remove repeate interleave and local scatter · 414a2f86
  Rick Ho authored May 19, 2021
  
  414a2f86
- update test · f804a121
  Rick Ho authored May 19, 2021
  
  f804a121
18 May, 2021 2 commits
- use customized pos assignment kernel to support -1 · 980cf4b6
  Rick Ho authored May 18, 2021
  
  980cf4b6
- fix bugs to run on multiple gpus · a468db2b
  Rick Ho authored May 18, 2021
  
  a468db2b
13 May, 2021 3 commits
- test switch gate · 38b334cc
  Rich Ho authored May 13, 2021
  
  38b334cc
- gshard gate test · ddfaaf49
  Rich Ho authored May 13, 2021
  
  ddfaaf49
- add test but cannot pass · 5a0ba835
  Rick Ho authored May 13, 2021
  
  5a0ba835
05 May, 2021 1 commit
- New data placement structure, speedup · b0704f1d
  TiagoMAntunes authored May 05, 2021
  
  b0704f1d
03 May, 2021 1 commit
- New simplified column reduction kernel · f165928a
  TiagoMAntunes authored May 04, 2021
  
  f165928a
29 Apr, 2021 1 commit
- Moved test into numerical. Handled different precision for half tensors · 69151519
  TiagoMAntunes authored Apr 29, 2021
  
  69151519
28 Apr, 2021 1 commit
- Added test for multiple data types in forward and backward pass · f5408e3c
  TiagoMAntunes authored Apr 28, 2021
  
  f5408e3c
27 Apr, 2021 6 commits
- Fixed incorrect shared memory type in column reduce kernel · 03b78901
  TiagoMAntunes authored Apr 27, 2021
  
  03b78901
- gshard prune gate cuda code and test · 56cb8c15
  Rick Ho authored Apr 27, 2021
  
  56cb8c15
- compute capacity in gshard · 3e8c263c
  Rick Ho authored Apr 27, 2021
  
  3e8c263c
- balancing cuda code · 1aced6d8
  Rick Ho authored Apr 27, 2021
  
  1aced6d8
- global exchange update variable name · 6cb6bbe4
  Rick Ho authored Apr 27, 2021
  
  6cb6bbe4
- limit by threshold cuda code · bb92d30e
  Rick Ho authored Apr 27, 2021
  
  bb92d30e