GitLab

- Split a model's computation into multiple virtual stages as needed,
and schedule communication correctly between these virtual stages
- Move schedule code into `schedules.py` and communication code into
`p2p_communication.py`
- Use hyphens instead of spaces in all time logging for consistency
- Factor out code in megatron/training.py into helper functions
- Refactor evaluate() function: make it use forward_backward_schedule
functions

Checkpoint should be saved only after evaluation pass is run to make sure validation losses are identical after loading checkpoint

Add the option for fp32 residual connection (fp32 residual connection machinery still needs to be added)