Commits · fa023f32fbf24a0cac4f733ff876188ccf193a8f · OpenDAS / FastMoE

24 Feb, 2021 2 commits
- allow user to indicate number of experts and topk · fa023f32
  Jiezhong Qiu authored Feb 24, 2021
  
  fa023f32
- fixes pytorch/pytorch/issues/36035 · d7706f0d
  Jiezhong Qiu authored Feb 24, 2021
```
follow suggestions from https://github.com/pytorch/pytorch/issues/36035#issuecomment-770960405
```
  d7706f0d
07 Feb, 2021 1 commit
- update the mem-transformer example · fb3e3c29
  Rick Ho authored Feb 07, 2021
  
  fb3e3c29
05 Feb, 2021 1 commit
- fit fmoe in transformer-xl · 9c92be55
  Rick Ho authored Feb 05, 2021
  
  9c92be55
25 Jan, 2021 1 commit
- move original files to examples · 0c47afbb
  Rick Ho authored Jan 25, 2021
  
  0c47afbb
13 Jan, 2021 2 commits
- merge topk and only forward moe once · 969ef607
  Jiezhong Qiu authored Jan 13, 2021
  
  969ef607
- import from pytorch dir · 6b38250f
  Jiezhong Qiu authored Jan 13, 2021
  
  6b38250f
03 Jan, 2021 2 commits
- 2 out of 32 experts · 6429fa29
  Jiezhong Qiu authored Jan 03, 2021
  
  6429fa29
- can train now · 97a2b1be
  Jiezhong Qiu authored Jan 03, 2021
  
  97a2b1be
02 Jan, 2021 3 commits
- fix bug · 7945bce6
  Jiezhong Qiu authored Jan 03, 2021
  
  7945bce6
- CUDA error at moe_cuda_kernel.cu:86 code=13(CUBLAS_STATUS_EXECUTION_FAILED) · 85306aa7
  Jiezhong Qiu authored Jan 03, 2021
  
  85306aa7
- incoporate customized cuda moe input xfmr-xl · 2338a26e
  Jiezhong Qiu authored Jan 03, 2021
  
  2338a26e
16 Dec, 2020 1 commit
- update · 2fef9391
  Jiezhong Qiu authored Dec 16, 2020
  
  2fef9391
08 Dec, 2020 1 commit
- multihead moe · b1dd8572
  Jiezhong Qiu authored Dec 08, 2020
  
  b1dd8572
07 Dec, 2020 3 commits
- update · 510ac924
  Jiezhong Qiu authored Dec 07, 2020
  
  510ac924
- by default using 1 block · 39996fef
  Jiezhong Qiu authored Dec 07, 2020
  
  39996fef
- customized topk · 1feaaf0c
  Jiezhong Qiu authored Dec 07, 2020
```
* when k=1, it reduces to torch.max, and not surprising that torch.max is
faster than torch.topk.
* however when k=2, it is even slower than torch.topk
```
  1feaaf0c
06 Dec, 2020 3 commits
- not use torch sparse anymore · fc279894
  Jiezhong Qiu authored Dec 07, 2020
  
  fc279894
- softmax rather than relu · 9f8d8cd8
  Jiezhong Qiu authored Dec 07, 2020
  
  9f8d8cd8
- residual in hmoe · 51361a3f
  Jiezhong Qiu authored Dec 06, 2020
  
  51361a3f
25 Nov, 2020 1 commit
- recover sparse ffn · 823f9c2e
  Jiezhong Qiu authored Nov 25, 2020
  
  823f9c2e
23 Nov, 2020 5 commits
- add gate · 0f3e63eb
  Jiezhong Qiu authored Nov 23, 2020
  
  0f3e63eb
- update · a43caff7
  Jiezhong Qiu authored Nov 23, 2020
  
  a43caff7
- fix · 4f7a182b
  Jiezhong Qiu authored Nov 23, 2020
  
  4f7a182b
- update · 103c6395
  Jiezhong Qiu authored Nov 23, 2020
  
  103c6395
- hmoe · 19ee0ff2
  Jiezhong Qiu authored Nov 23, 2020
  
  19ee0ff2
19 Nov, 2020 3 commits
- fix · 3bdbb28d
  Jiezhong Qiu authored Nov 19, 2020
  
  3bdbb28d
- torch sparse for spmm · 1b7cbeb5
  Jiezhong Qiu authored Nov 19, 2020
  
  1b7cbeb5
- fix bug · 47167bcc
  Jiezhong Qiu authored Nov 19, 2020
  
  47167bcc
18 Nov, 2020 2 commits
- fix · 82fe21d3
  Jiezhong Qiu authored Nov 19, 2020
  
  82fe21d3
- multihead ffn · 37d01e9c
  Jiezhong Qiu authored Nov 18, 2020
  
  37d01e9c
14 Nov, 2020 1 commit
- profile sparsity of relu output in ffn · cf8a61d8
  Jiezhong Qiu authored Nov 14, 2020
  
  cf8a61d8
08 Nov, 2020 1 commit
- add mem2mem attention · 958c714e
  Jiezhong Qiu authored Nov 08, 2020
  
  958c714e
09 Jan, 2019 1 commit
- init · 3a0086fa
  Zhilin Yang authored Jan 10, 2019
  
  3a0086fa