Commits · c931c484cb4cd4de57b79325f48b092fed3e1c91 · OpenDAS / FastMoE

20 May, 2021 4 commits
- fix zero test · c931c484
  Rich Ho authored May 20, 2021
  
  c931c484
- Merge branch 'master' into laekov/gate · 5680c599
  Rich Ho authored May 20, 2021
  
  5680c599
- Merge pull request #21 from bias_improvement · 3c42c892
  Rick Ho authored May 20, 2021
```
Bias improvement #15
```
  3c42c892
- add data_type in test ddp · 41cfe06c
  Rich Ho authored May 20, 2021
  
  41cfe06c
19 May, 2021 5 commits
- fix scatter bug across gpus · 90c4bccf
  Rich Ho authored May 19, 2021
  
  90c4bccf
- pass test on single gpu · 670d70ed
  Rick Ho authored May 19, 2021
  
  670d70ed
- remove local scatter and gather in cuda · 52e57316
  Rick Ho authored May 19, 2021
  
  52e57316
- remove repeate interleave and local scatter · 414a2f86
  Rick Ho authored May 19, 2021
  
  414a2f86
- update test · f804a121
  Rick Ho authored May 19, 2021
  
  f804a121
18 May, 2021 2 commits
- use customized pos assignment kernel to support -1 · 980cf4b6
  Rick Ho authored May 18, 2021
  
  980cf4b6
- fix bugs to run on multiple gpus · a468db2b
  Rick Ho authored May 18, 2021
  
  a468db2b
13 May, 2021 3 commits
- test switch gate · 38b334cc
  Rich Ho authored May 13, 2021
  
  38b334cc
- gshard gate test · ddfaaf49
  Rich Ho authored May 13, 2021
  
  ddfaaf49
- add test but cannot pass · 5a0ba835
  Rick Ho authored May 13, 2021
  
  5a0ba835
05 May, 2021 1 commit
- New data placement structure, speedup · b0704f1d
  TiagoMAntunes authored May 05, 2021
  
  b0704f1d
03 May, 2021 1 commit
- New simplified column reduction kernel · f165928a
  TiagoMAntunes authored May 04, 2021
  
  f165928a
29 Apr, 2021 1 commit
- Moved test into numerical. Handled different precision for half tensors · 69151519
  TiagoMAntunes authored Apr 29, 2021
  
  69151519
28 Apr, 2021 1 commit
- Added test for multiple data types in forward and backward pass · f5408e3c
  TiagoMAntunes authored Apr 28, 2021
  
  f5408e3c
27 Apr, 2021 6 commits
- Fixed incorrect shared memory type in column reduce kernel · 03b78901
  TiagoMAntunes authored Apr 27, 2021
  
  03b78901
- gshard prune gate cuda code and test · 56cb8c15
  Rick Ho authored Apr 27, 2021
  
  56cb8c15
- compute capacity in gshard · 3e8c263c
  Rick Ho authored Apr 27, 2021
  
  3e8c263c
- balancing cuda code · 1aced6d8
  Rick Ho authored Apr 27, 2021
  
  1aced6d8
- global exchange update variable name · 6cb6bbe4
  Rick Ho authored Apr 27, 2021
  
  6cb6bbe4
- limit by threshold cuda code · bb92d30e
  Rick Ho authored Apr 27, 2021
  
  bb92d30e
26 Apr, 2021 5 commits
- fix zero gate and constant gate · 450549a4
  Rick Ho authored Apr 26, 2021
  
  450549a4
- fix stream manager · 956d3aef
  Rick Ho authored Apr 26, 2021
  
  956d3aef
- reconstruct fmoe cuda code · bb3c8966
  Rick Ho authored Apr 26, 2021
  
  bb3c8966
- add gshard and switch gate with loss · 4c90e6e8
  Rick Ho authored Apr 26, 2021
  
  4c90e6e8
- reconstruct gate structure · 82103edb
  Rick Ho authored Apr 26, 2021
  
  82103edb
04 Apr, 2021 1 commit
- New compute kernel for column reduction · 3d1987d1
  TiagoMAntunes authored Apr 04, 2021
  
  3d1987d1
29 Mar, 2021 1 commit
- Merged with master · f957c299
  TiagoMAntunes authored Mar 29, 2021
  
  f957c299
28 Mar, 2021 2 commits
- Fixed indentation · 7001bc3e
  TiagoMAntunes authored Mar 29, 2021
  
  7001bc3e
- Added CUDA kernel for column reduction · 01d888bf
  TiagoMAntunes authored Mar 29, 2021
  
  01d888bf
25 Mar, 2021 6 commits
- Bias moved to CUDA in forward. Basic bias setup for backwards (no kernel yet) · c5f73a0f
  TiagoMAntunes authored Mar 26, 2021
  
  c5f73a0f
- Remove wierd line break in bib · 26824495
  Rick Ho authored Mar 25, 2021
  
  26824495
- Merge pull request #23 from laekov/laekov/add-citation · 46c0574b
  Jiezhong Qiu authored Mar 25, 2021
```
citation from arxiv
```
  46c0574b
- fix bibtex format · 1f5c1ce1
  Rick Ho authored Mar 25, 2021
  
  1f5c1ce1
- Merge pull request #22 from laekov/laekov/fix-tests · d81cb62f
  Jiezhong Qiu authored Mar 25, 2021
```
fix tests after updating megatron
```
  d81cb62f
- citation from arxiv · 3839d90f
  Rick Ho authored Mar 25, 2021
  
  3839d90f
23 Mar, 2021 1 commit
- Fixed indentation (4 spaces now) · 6cdb3cda
  TiagoMAntunes authored Mar 24, 2021
  
  6cdb3cda