Commits · 3a41edb8c29eae02fd286302ab85d809c73fff18 · OpenDAS / FastMoE

11 Sep, 2023 2 commits
- fix shadow · 945004e7
  Rick Ho authored Sep 11, 2023
  
  945004e7
- use torchstream everywhere · 4f9f77f8
  Rick Ho authored Sep 11, 2023
  
  4f9f77f8
02 Sep, 2023 1 commit
- [BUG FIX] sync torch stream before nccl send/recv · 2bd187cb
  zms1999 authored Sep 02, 2023
  
  2bd187cb
25 Aug, 2023 2 commits
- [BUG FIX] wait torch stream · ff28081c
  zms1999 authored Aug 25, 2023
  
  ff28081c
- [BUG FIX] make smart scheduling great again, fix bugs in streams management · c8633740
  zms1999 authored Aug 25, 2023
  
  c8633740
06 Jun, 2023 1 commit
- bf16 support with test · 0bb60881
  Rick Ho authored Jun 06, 2023
  
  0bb60881
21 Mar, 2023 1 commit
- support n_expert > 1 for FasterMoE smart scheduling and expert shadowing · 14878015
  zms1999 authored Mar 21, 2023
  
  14878015
20 Mar, 2023 1 commit
- patch for megatron v2.5 · b31b7e3f
  zms1999 authored Mar 21, 2023
  
  b31b7e3f
28 Dec, 2022 1 commit
- cudaStreamWaitEvent compat · 25e9324f
  Rick Ho authored Dec 28, 2022
  
  25e9324f
30 Apr, 2022 1 commit
- skip computeFn when micro_batch_size == 0 · 2ce87038
  zms1999 authored Apr 30, 2022
  
  2ce87038
31 Mar, 2022 1 commit
- expert shadow backward with test · 794dd0e6
  Rick Ho authored Mar 31, 2022
  
  794dd0e6
30 Mar, 2022 3 commits
- forward tested · b5b72d41
  Rick Ho authored Mar 30, 2022
  
  b5b72d41
- forward code in smart schedule · 771dc62d
  Rick Ho authored Mar 30, 2022
  
  771dc62d
- fix potential stream synchronization issue · ad651f03
  Rick Ho authored Mar 30, 2022
  
  ad651f03
29 Mar, 2022 3 commits
- fix backward grad weight bug · 6c68b56b
  Rick Ho authored Mar 30, 2022
  
  6c68b56b
- backward bugous on grad weight · a807e2a3
  Rick Ho authored Mar 29, 2022
  
  a807e2a3
- forward pass test · 46c3722d
  Rick Ho authored Mar 29, 2022
  
  46c3722d