"megatron/legacy/model/realm_model.py" did not exist on "2f7d666cf453bdd9afb085e9c9a10868b3c0af05"
-
Deepak Narayanan authored
Also includes following changes for inter-layer model-parallel implementation: - Refactoring of model implementations - Training loop changes to support inter-layer communication using `ring_exchange` - New groups for inter-layer communication - Checkpoint changes - Command line arguments
7abd3e90
