Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
FastMoE
Commits
56c1bd63b77470168c0aaafd4fce264ebe4595ca
Switch branch/tag
fastmoe
fmoe
megatron.py
04 Feb, 2021
3 commits
fix no grad after all-gather bug
· 56c1bd63
Rick Ho
authored
Feb 04, 2021
56c1bd63
use parallel label in gate
· d83234b0
Rick Ho
authored
Feb 04, 2021
d83234b0
ddp module for sophiscated hybrid parallel
· 67c667f2
Rick Ho
authored
Feb 04, 2021
67c667f2
03 Feb, 2021
2 commits
fix pure data parallel
· ae2c434e
Rick Ho
authored
Feb 03, 2021
ae2c434e
fmoefy
· 6b8d2f2e
Rick Ho
authored
Feb 03, 2021
6b8d2f2e
02 Feb, 2021
3 commits
fix replica condition and minor optimizations
· dc3db673
Rick Ho
authored
Feb 02, 2021
dc3db673
Optimize redundancy communication
· 01ae2d72
Sengxian
authored
Feb 02, 2021
01ae2d72
Format using black and add model_parallel_rank
· fdbac1df
Sengxian
authored
Feb 02, 2021
fdbac1df
29 Jan, 2021
2 commits
fix python bugs
· 6900f1de
Rick Ho
authored
Jan 29, 2021
6900f1de
reconstruct fmoe nn module
· 437afda2
Rick Ho
authored
Jan 29, 2021
437afda2
26 Jan, 2021
1 commit
initial version to run with megatron
· 58e949cf
Rick Ho
authored
Jan 26, 2021
58e949cf
25 Jan, 2021
1 commit
basic megatron support frame
· d0f07ff7
Rick Ho
authored
Jan 25, 2021
d0f07ff7
