Commits · 8052b3c0eaff8bde0b427ffcd8f514cd083e1411 · OpenDAS / FastMoE

26 Oct, 2021 1 commit
- cpu swipe · 8052b3c0
  Rick Ho authored Oct 26, 2021
  
  8052b3c0
30 Sep, 2021 5 commits
- Update helper_cuda.h · 8ebdc789
  hclearner authored Sep 30, 2021
  
  8ebdc789
- Update cublas_wrapper.h · 9d0f67d6
  hclearner authored Sep 30, 2021
  
  9d0f67d6
- Update cublas_wrapper.h · 43eabe75
  hclearner authored Sep 30, 2021
  
  43eabe75
- Update local_exchange.cuh · 976ef2b8
  hclearner authored Sep 30, 2021
  
  976ef2b8
- add fastmoe support rocm4.0.1 · 2a77b772
  huchen1 authored Sep 30, 2021
  
  2a77b772
28 Jun, 2021 2 commits
- acro bug fix · 9170835c
  Rich Ho authored Jun 28, 2021
  
  9170835c
- fix pytorch compatibility issue !55 · 17e2a5c5
  Rick Ho authored Jun 28, 2021
  
  17e2a5c5
16 Jun, 2021 1 commit

Improve efficiency of metadata exchange (#48) · 295a615a

Rick Ho authored Jun 16, 2021

* use single variable instead of vector in c functions

* expert count kernel

* remove all lists

* fix old tests

295a615a

09 Jun, 2021 1 commit
- Fixed asynchronous streams in column reduce kernel call · 2126b59a
  TiagoMAntunes authored Jun 09, 2021
  
  2126b59a
29 May, 2021 1 commit
- fix bugs to run megatron with gshard gate · fa5f45f0
  Rick Ho authored May 29, 2021
  
  fa5f45f0
21 May, 2021 1 commit
- update accoding to comments · 8d14dd29
  Rich Ho authored May 21, 2021
  
  8d14dd29
19 May, 2021 2 commits
- pass test on single gpu · 670d70ed
  Rick Ho authored May 19, 2021
  
  670d70ed
- remove local scatter and gather in cuda · 52e57316
  Rick Ho authored May 19, 2021
  
  52e57316
18 May, 2021 1 commit
- use customized pos assignment kernel to support -1 · 980cf4b6
  Rick Ho authored May 18, 2021
  
  980cf4b6
13 May, 2021 1 commit
- add test but cannot pass · 5a0ba835
  Rick Ho authored May 13, 2021
  
  5a0ba835
05 May, 2021 1 commit
- New data placement structure, speedup · b0704f1d
  TiagoMAntunes authored May 05, 2021
  
  b0704f1d
03 May, 2021 1 commit
- New simplified column reduction kernel · f165928a
  TiagoMAntunes authored May 04, 2021
  
  f165928a
27 Apr, 2021 5 commits
- Fixed incorrect shared memory type in column reduce kernel · 03b78901
  TiagoMAntunes authored Apr 27, 2021
  
  03b78901
- gshard prune gate cuda code and test · 56cb8c15
  Rick Ho authored Apr 27, 2021
  
  56cb8c15
- balancing cuda code · 1aced6d8
  Rick Ho authored Apr 27, 2021
  
  1aced6d8
- global exchange update variable name · 6cb6bbe4
  Rick Ho authored Apr 27, 2021
  
  6cb6bbe4
- limit by threshold cuda code · bb92d30e
  Rick Ho authored Apr 27, 2021
  
  bb92d30e
26 Apr, 2021 2 commits
- fix stream manager · 956d3aef
  Rick Ho authored Apr 26, 2021
  
  956d3aef
- reconstruct fmoe cuda code · bb3c8966
  Rick Ho authored Apr 26, 2021
  
  bb3c8966
04 Apr, 2021 1 commit
- New compute kernel for column reduction · 3d1987d1
  TiagoMAntunes authored Apr 04, 2021
  
  3d1987d1
28 Mar, 2021 2 commits
- Fixed indentation · 7001bc3e
  TiagoMAntunes authored Mar 29, 2021
  
  7001bc3e
- Added CUDA kernel for column reduction · 01d888bf
  TiagoMAntunes authored Mar 29, 2021
  
  01d888bf
25 Mar, 2021 1 commit
- Bias moved to CUDA in forward. Basic bias setup for backwards (no kernel yet) · c5f73a0f
  TiagoMAntunes authored Mar 26, 2021
  
  c5f73a0f
23 Mar, 2021 2 commits
- Fixed indentation (4 spaces now) · 6cdb3cda
  TiagoMAntunes authored Mar 24, 2021
  
  6cdb3cda
- Bias now being calculated directly in MOELinear layer. Added corresponding... · 303d0e93
  TiagoMAntunes authored Mar 23, 2021
```
Bias now being calculated directly in MOELinear layer. Added corresponding CUDA changes. Updated forward and backward functions of MOELinear
```
  303d0e93
22 Mar, 2021 1 commit
- Add gshard and noisy gate balance strategy · 89d6c794
  Sengxian authored Mar 22, 2021
  
  89d6c794
13 Mar, 2021 1 commit
- manually broadcast nccl unique id · 43af1522
  Rick Ho authored Mar 13, 2021
  
  43af1522
04 Mar, 2021 1 commit
- remove dependency on cuda sample repo · 7d695acb
  Rick Ho authored Mar 04, 2021
  
  7d695acb
07 Feb, 2021 1 commit
- strong compatibility to different python and pytorch versions · 103343ca
  Rick Ho authored Feb 07, 2021
  
  103343ca
04 Feb, 2021 2 commits
- fix pytorch header compilation bug · 79ccb7b6
  Rick Ho authored Feb 04, 2021
  
  79ccb7b6
- adapt with pytorch 1.8.0 (deprecated 1.6.0) · 15f98a10
  Rick Ho authored Feb 04, 2021
  
  15f98a10
03 Feb, 2021 1 commit
- fix ensure device index bug · ea66e5e5
  Rick Ho authored Feb 03, 2021
  
  ea66e5e5
01 Feb, 2021 1 commit
- swap local scatter and gather kernel functions · d2039fc7
  Rick Ho authored Feb 01, 2021
  
  d2039fc7
29 Jan, 2021 1 commit
- scatter gather kernel support for non-equal shapes and fix tests · 72e9bc9e
  Rick Ho authored Jan 29, 2021
  
  72e9bc9e