GitLab

consider the case of pipeline-model prallelism

clean up arugments

argument naming cleanup

update readme and examples

The --seq-length argument was missing from the example distributed BERT
pre-training script which throws an error while attempting to parse the
arguments as the code expects all arguments to be specified and have a
non-None value.
Signed-Off-By: Robert Clark <roclark@nvidia.com>

remove erroneous arg.

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments