Commits · f957c299a115a69464ea02d4e0b102556dc2767b · OpenDAS / FastMoE

29 Mar, 2021 1 commit
- Merged with master · f957c299
  TiagoMAntunes authored Mar 29, 2021
  
  f957c299
28 Mar, 2021 2 commits
- Fixed indentation · 7001bc3e
  TiagoMAntunes authored Mar 29, 2021
  
  7001bc3e
- Added CUDA kernel for column reduction · 01d888bf
  TiagoMAntunes authored Mar 29, 2021
  
  01d888bf
25 Mar, 2021 6 commits
- Bias moved to CUDA in forward. Basic bias setup for backwards (no kernel yet) · c5f73a0f
  TiagoMAntunes authored Mar 26, 2021
  
  c5f73a0f
- Remove wierd line break in bib · 26824495
  Rick Ho authored Mar 25, 2021
  
  26824495
- Merge pull request #23 from laekov/laekov/add-citation · 46c0574b
  Jiezhong Qiu authored Mar 25, 2021
```
citation from arxiv
```
  46c0574b
- fix bibtex format · 1f5c1ce1
  Rick Ho authored Mar 25, 2021
  
  1f5c1ce1
- Merge pull request #22 from laekov/laekov/fix-tests · d81cb62f
  Jiezhong Qiu authored Mar 25, 2021
```
fix tests after updating megatron
```
  d81cb62f
- citation from arxiv · 3839d90f
  Rick Ho authored Mar 25, 2021
  
  3839d90f
23 Mar, 2021 4 commits
- Fixed indentation (4 spaces now) · 6cdb3cda
  TiagoMAntunes authored Mar 24, 2021
  
  6cdb3cda
- fix tests after updating megatron · 6868ed2a
  Rick Ho authored Mar 24, 2021
  
  6868ed2a
- Bias now being calculated directly in MOELinear layer. Added corresponding... · 303d0e93
  TiagoMAntunes authored Mar 23, 2021
```
Bias now being calculated directly in MOELinear layer. Added corresponding CUDA changes. Updated forward and backward functions of MOELinear
```
  303d0e93
- Updated arguments for MOELinear.apply · 8bac18dc
  TiagoMAntunes authored Mar 23, 2021
  
  8bac18dc
22 Mar, 2021 27 commits
- Merge pull request #20 from laekov/laekov/split-megatron · cac233f3
  Rick Ho authored Mar 22, 2021
```
Split megatron python module
```
  cac233f3
- add fastmoe.megatron to install list · 9a4434f5
  Rick Ho authored Mar 22, 2021
  
  9a4434f5
- add balance to init · 2bceb76b
  Rick Ho authored Mar 22, 2021
  
  2bceb76b
- initial split · b690f245
  Rick Ho authored Mar 22, 2021
  
  b690f245
- Merge branch 'master' of github.com:laekov/fastmoe · 66fc166e
  Rick Ho authored Mar 22, 2021
  
  66fc166e
- Revise variable name · 1b7817a5
  Sengxian authored Mar 22, 2021
  
  1b7817a5
- Update issue templates · 038b31ea
  Rick Ho authored Mar 22, 2021
  
  038b31ea
- remove debug info · bedaa47d
  Jiezhong Qiu authored Mar 22, 2021
  
  bedaa47d
- format with black · 6b5a34cc
  Jiezhong Qiu authored Mar 22, 2021
  
  6b5a34cc
- relax max line length to 120 · cf9fd12a
  Jiezhong Qiu authored Mar 22, 2021
  
  cf9fd12a
- fix pylint · 68997976
  Jiezhong Qiu authored Mar 22, 2021
  
  68997976
- load/save fp32 main weight when fp16 training · 90e394bc
  Jiezhong Qiu authored Mar 22, 2021
  
  90e394bc
- fp16/fp32 optimizer has different data structure · bcfeaf3b
  Jiezhong Qiu authored Mar 17, 2021
  
  bcfeaf3b
- remove unnecessary states when dp rank>0 · 12b23ae3
  Jiezhong Qiu authored Mar 17, 2021
  
  12b23ae3
- Create Megatron.LICENSE · bc655118
  Jiezhong Qiu authored Mar 17, 2021
  
  bc655118
- load checkpoint for expert parallel · 24541d6a
  Jiezhong Qiu authored Mar 17, 2021
  
  24541d6a
- avoid saveing duplicated optim state in dp rank>0 · f5a5d31a
  Jiezhong Qiu authored Mar 17, 2021
  
  f5a5d31a
- fix pylint · c844413b
  Jiezhong Qiu authored Mar 16, 2021
  
  c844413b
- save experts separately in each data parallel rank · 49a4678c
  Jiezhong Qiu authored Mar 16, 2021
  
  49a4678c
- Merge pull request #19 from laekov/balance · 3c2a5979
  Rick Ho authored Mar 22, 2021
```
Add balance strategy
```
  3c2a5979
- Revise variable name · e028f2ec
  Sengxian authored Mar 22, 2021
  
  e028f2ec
- Merge remote-tracking branch 'origin/master' into balance_public · 69121432
  Sengxian authored Mar 22, 2021
  
  69121432
- Merge pull request #18 from laekov/checkpoint · c1e67585
  Rick Ho authored Mar 22, 2021
```
Checkpoint
```
  c1e67585
- Merge remote-tracking branch 'origin/master' into checkpoint · d155925d
  Jiezhong Qiu authored Mar 22, 2021
```
Conflicts:
	fmoe/transformer.py
```
  d155925d
- Add gshard and noisy gate balance strategy · 89d6c794
  Sengxian authored Mar 22, 2021
  
  89d6c794
- Update issue templates · 98b4b0be
  Rick Ho authored Mar 22, 2021
  
  98b4b0be
- remove debug info · f49a9ec4
  Jiezhong Qiu authored Mar 22, 2021
  
  f49a9ec4