Commits · e70726a923c693dacd18d9a5df00e3ce8f18d54f · OpenDAS / FastMoE

24 Jul, 2023 2 commits
- Parallelism Document · e70726a9
  Rick Ho authored Jul 24, 2023
  
  e70726a9
- update readme and inline docstring · 02341276
  Rick Ho authored Jul 24, 2023
  
  02341276
04 Jul, 2023 1 commit
- Megatron v3.0.2 Patch (#159) · a6d202a6
  Jiezhong Qiu authored Jul 04, 2023
```
* support megatron v3.0.2

* keep num_experts for lower version of megatron
```
  a6d202a6
06 Jun, 2023 2 commits
- Merge pull request #160 from laekov/nvbf16 · d56522bc
  Mingshu Zhai authored Jun 06, 2023
```
bf16 support
```
  d56522bc
- bf16 support with test · 0bb60881
  Rick Ho authored Jun 06, 2023
  
  0bb60881
30 May, 2023 2 commits
- release note v1.0.1 · c9ccc0eb
  Rick Ho authored May 30, 2023
  
  c9ccc0eb
- Merge pull request #157 from laekov/pytorch2-compat · 1e8e455a
  Rick Ho authored May 30, 2023
```
Fix ProcessGroupNCCL mismatch in pytorch2
```
  1e8e455a
29 May, 2023 1 commit
- fix processgroupnccl mismatch in pytorch2 · 267eb9cc
  Rick Ho authored May 29, 2023
  
  267eb9cc
21 May, 2023 5 commits
- Merge pull request #155 from santurini/patch-3 · 9fe65ec0
  Rick Ho authored May 21, 2023
```
Added link to installation guide
```
  9fe65ec0
- Merge pull request #154 from santurini/patch-2 · 2ec2f14a
  Rick Ho authored May 21, 2023
```
Create installation-guide.md
```
  2ec2f14a
- Added link to installation guide · 7817fa15
  Arturo Ghinassi authored May 21, 2023
  
  7817fa15
- Reviewed following @laekov suggestions · 3eabb02f
  Arturo Ghinassi authored May 21, 2023
  
  3eabb02f
- Create installation-guide.md · cb74e2f6
  Arturo Ghinassi authored May 21, 2023
```
Added installation tutorial in the documentation
```
  cb74e2f6
19 May, 2023 6 commits
- Merge pull request #152 from santurini/patch-1 · 50ac47f9
  Rick Ho authored May 19, 2023
```
Cast input to weights type for AMP support
```
  50ac47f9
- Cast input to weights type for AMP support · af8c4737
  Arturo Ghinassi authored May 19, 2023
  
  af8c4737
- Merge pull request #151 from laekov/revert-150-patch-1 · a31fcc55
  Rick Ho authored May 19, 2023
```
Revert "convert input to same type as weight for mixed precision training"
```
  a31fcc55
- Revert "convert input to same type as weight for mixed precision training" · f1a6e6fd
  Rick Ho authored May 19, 2023
  
  f1a6e6fd
- Merge pull request #150 from santurini/patch-1 · 3a0eb3e6
  Rick Ho authored May 19, 2023
```
convert input to same type as weight for mixed precision training
```
  3a0eb3e6
- changed .float() to .as_type(self.weight) · 1f7c04c9
  Arturo Ghinassi authored May 19, 2023
```
AMP support
```
  1f7c04c9
18 May, 2023 1 commit
- convert input to same type as weight for mixed precision training · 2196b766
  Arturo Ghinassi authored May 18, 2023
```
When using CUDA AMP FMoE Linear throws type error as input is half() and weights are float()
```
  2196b766
21 Mar, 2023 3 commits
- Merge pull request #146 from laekov/fastermoe_multi_experts · df715c9f
  Rick Ho authored Mar 21, 2023
```
make FasterMoE more general
```
  df715c9f
- support n_expert > 1 for FasterMoE smart scheduling and expert shadowing · 14878015
  zms1999 authored Mar 21, 2023
  
  14878015
- Merge pull request #145 from laekov/patch_megatron_v2_5 · 698a12ae
  Rick Ho authored Mar 21, 2023
```
FastMoE with Megatron-LM v2.5
```
  698a12ae
20 Mar, 2023 1 commit
- patch for megatron v2.5 · b31b7e3f
  zms1999 authored Mar 21, 2023
  
  b31b7e3f
13 Feb, 2023 2 commits
- Merge pull request #144 from Fragile-azalea/master · b8d0e81f
  Rick Ho authored Feb 13, 2023
```
remove synchronize
```
  b8d0e81f
- remove synchronize · 6ebec59c
  Fragile-azalea authored Feb 13, 2023
  
  6ebec59c
28 Dec, 2022 3 commits
- Merge pull request #142 from laekov/torch-1.13-compat · 9303ac18
  Rick Ho authored Dec 28, 2022
```
Compatibility to older cuda and torch 1.13
```
  9303ac18
- cudaStreamWaitEvent compat · 25e9324f
  Rick Ho authored Dec 28, 2022
  
  25e9324f
- update header file processgroupnccl · c7e6a3db
  Rick Ho authored Dec 28, 2022
  
  c7e6a3db
14 Dec, 2022 3 commits
- Merge pull request #141 from laekov/gate-fix · 2cb96b40
  Rick Ho authored Dec 14, 2022
```
Diverge gshard gate
```
  2cb96b40
- capacity numerical fix · a762d33c
  Rick Ho authored Dec 14, 2022
  
  a762d33c
- diverge gshard gate · 6cb550fd
  Rick Ho authored Dec 14, 2022
  
  6cb550fd
14 Nov, 2022 2 commits
- Merge pull request #138 from laekov/clip-bug · cdc140f1
  Rick Ho authored Nov 14, 2022
```
fix bug: add proper comm group
```
  cdc140f1
- fix bug: add proper comm group · 66d27e9a
  zms1999 authored Nov 14, 2022
  
  66d27e9a
30 Sep, 2022 2 commits
- Merge pull request #136 from laekov/update-versions · 53844b65
  Rick Ho authored Sep 30, 2022
```
Update version requirement in the documents
```
  53844b65
- update version requirement in the documents · d9b6359d
  Rick Ho authored Sep 30, 2022
  
  d9b6359d
19 Sep, 2022 1 commit
- Merge pull request #135 from laekov/example-doc · 53bac444
  Rick Ho authored Sep 19, 2022
```
Document for examples
```
  53bac444
16 Sep, 2022 1 commit
- document for examples · 8cbe171d
  Rick Ho authored Sep 16, 2022
  
  8cbe171d
04 Aug, 2022 2 commits
- Merge pull request #130 from Fragile-azalea/master · 40f5fd36
  Rick Ho authored Aug 04, 2022
```
Fix GshardGate top1_idx
```
  40f5fd36
- fix GshardGate top1_idx · 5f8e3999
  Fragile-azalea authored Aug 04, 2022
  
  5f8e3999