Commits · cb74e2f61e1871db5cd9a00ca0da29d69c31b5be · OpenDAS / FastMoE

21 May, 2023 1 commit
- Create installation-guide.md · cb74e2f6
  Arturo Ghinassi authored May 21, 2023
```
Added installation tutorial in the documentation
```
  cb74e2f6
19 May, 2023 6 commits
- Merge pull request #152 from santurini/patch-1 · 50ac47f9
  Rick Ho authored May 19, 2023
```
Cast input to weights type for AMP support
```
  50ac47f9
- Cast input to weights type for AMP support · af8c4737
  Arturo Ghinassi authored May 19, 2023
  
  af8c4737
- Merge pull request #151 from laekov/revert-150-patch-1 · a31fcc55
  Rick Ho authored May 19, 2023
```
Revert "convert input to same type as weight for mixed precision training"
```
  a31fcc55
- Revert "convert input to same type as weight for mixed precision training" · f1a6e6fd
  Rick Ho authored May 19, 2023
  
  f1a6e6fd
- Merge pull request #150 from santurini/patch-1 · 3a0eb3e6
  Rick Ho authored May 19, 2023
```
convert input to same type as weight for mixed precision training
```
  3a0eb3e6
- changed .float() to .as_type(self.weight) · 1f7c04c9
  Arturo Ghinassi authored May 19, 2023
```
AMP support
```
  1f7c04c9
18 May, 2023 1 commit
- convert input to same type as weight for mixed precision training · 2196b766
  Arturo Ghinassi authored May 18, 2023
```
When using CUDA AMP FMoE Linear throws type error as input is half() and weights are float()
```
  2196b766
21 Mar, 2023 3 commits
- Merge pull request #146 from laekov/fastermoe_multi_experts · df715c9f
  Rick Ho authored Mar 21, 2023
```
make FasterMoE more general
```
  df715c9f
- support n_expert > 1 for FasterMoE smart scheduling and expert shadowing · 14878015
  zms1999 authored Mar 21, 2023
  
  14878015
- Merge pull request #145 from laekov/patch_megatron_v2_5 · 698a12ae
  Rick Ho authored Mar 21, 2023
```
FastMoE with Megatron-LM v2.5
```
  698a12ae
20 Mar, 2023 1 commit
- patch for megatron v2.5 · b31b7e3f
  zms1999 authored Mar 21, 2023
  
  b31b7e3f
13 Feb, 2023 2 commits
- Merge pull request #144 from Fragile-azalea/master · b8d0e81f
  Rick Ho authored Feb 13, 2023
```
remove synchronize
```
  b8d0e81f
- remove synchronize · 6ebec59c
  Fragile-azalea authored Feb 13, 2023
  
  6ebec59c
28 Dec, 2022 3 commits
- Merge pull request #142 from laekov/torch-1.13-compat · 9303ac18
  Rick Ho authored Dec 28, 2022
```
Compatibility to older cuda and torch 1.13
```
  9303ac18
- cudaStreamWaitEvent compat · 25e9324f
  Rick Ho authored Dec 28, 2022
  
  25e9324f
- update header file processgroupnccl · c7e6a3db
  Rick Ho authored Dec 28, 2022
  
  c7e6a3db
14 Dec, 2022 3 commits
- Merge pull request #141 from laekov/gate-fix · 2cb96b40
  Rick Ho authored Dec 14, 2022
```
Diverge gshard gate
```
  2cb96b40
- capacity numerical fix · a762d33c
  Rick Ho authored Dec 14, 2022
  
  a762d33c
- diverge gshard gate · 6cb550fd
  Rick Ho authored Dec 14, 2022
  
  6cb550fd
14 Nov, 2022 2 commits
- Merge pull request #138 from laekov/clip-bug · cdc140f1
  Rick Ho authored Nov 14, 2022
```
fix bug: add proper comm group
```
  cdc140f1
- fix bug: add proper comm group · 66d27e9a
  zms1999 authored Nov 14, 2022
  
  66d27e9a
30 Sep, 2022 2 commits
- Merge pull request #136 from laekov/update-versions · 53844b65
  Rick Ho authored Sep 30, 2022
```
Update version requirement in the documents
```
  53844b65
- update version requirement in the documents · d9b6359d
  Rick Ho authored Sep 30, 2022
  
  d9b6359d
19 Sep, 2022 1 commit
- Merge pull request #135 from laekov/example-doc · 53bac444
  Rick Ho authored Sep 19, 2022
```
Document for examples
```
  53bac444
16 Sep, 2022 1 commit
- document for examples · 8cbe171d
  Rick Ho authored Sep 16, 2022
  
  8cbe171d
04 Aug, 2022 2 commits
- Merge pull request #130 from Fragile-azalea/master · 40f5fd36
  Rick Ho authored Aug 04, 2022
```
Fix GshardGate top1_idx
```
  40f5fd36
- fix GshardGate top1_idx · 5f8e3999
  Fragile-azalea authored Aug 04, 2022
  
  5f8e3999
23 Jul, 2022 2 commits
- Merge pull request #127 from heheda12345/master · f2e0d1d5
  Rick Ho authored Jul 23, 2022
```
update readme: enable NCCL by default
```
  f2e0d1d5
- update readme: enable NCCL by default · 069da0e8
  heheda authored Jul 23, 2022
  
  069da0e8
18 Jul, 2022 2 commits
- Merge pull request #123 from laekov/torch-v1.12.0-compat · 5a5dfa18
  Rick Ho authored Jul 18, 2022
```
Fix nccl uid bcast for torch v1.12.0
```
  5a5dfa18
- fix nccl uid bcast · a0b1e31a
  Rick Ho authored Jul 18, 2022
  
  a0b1e31a
12 Jun, 2022 1 commit
- Merge pull request #120 from laekov/ddp-bcast-fix · 9a01f8fa
  Rick Ho authored Jun 12, 2022
```
Fix Broadcast rank bug in DGDP
```
  9a01f8fa
10 Jun, 2022 1 commit
- update ddp with get first rank and tests · dd68fd78
  Rick Ho authored Jun 10, 2022
  
  dd68fd78
01 Jun, 2022 4 commits
- Merge pull request #117 from laekov/distributed-doc · 670e1407
  Rick Ho authored Jun 01, 2022
```
Add a hint for DGDP synchronization
```
  670e1407
- add an option of init sync · 039b4550
  Rick Ho authored Jun 01, 2022
  
  039b4550
- fix bug for sync_params · 6fe415e9
  Rick Ho authored Jun 01, 2022
  
  6fe415e9
- add a hint for DGDP synchronization · 21a331fb
  Rick Ho authored Jun 01, 2022
  
  21a331fb
27 May, 2022 2 commits
- Merge pull request #115 from laekov/gate-test-fix · 527e66af
  Rick Ho authored May 27, 2022
```
Fix gshard gate test
```
  527e66af
- fix gshard gate test · e01c8b5f
  Rick Ho authored May 27, 2022
  
  e01c8b5f