GitLab

Fix document for megatron

Fix GShardGate random_routing

Fix naive gate output score

Fix missing positional argument in test_mimo.py

MoE with multiple inputs and multiple outputs

Fix transformer-XL examples

v0.3.0 Release

SWIPE balance strategy

Model parallelism of single experts

add fastmoe support rocm4.0.1