GitLab

Add the option for fp32 residual connection (fp32 residual connection machinery still needs to be added)

Small bugfix in bert_model.py: make sure word_embeddings is initialized before instantiating lm_head

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>