GitLab

Add the option for fp32 residual connection (fp32 residual connection machinery still needs to be added)

Rename --batch-size to --micro-batch-size and drop in-minibatch from --num-micro-batches-in-minibatch

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>