GitLab

Move rearranging query_key_value and key_value values in old checkpoints to when the checkpoint is loaded instead of runtime..

Rename --batch-size to --micro-batch-size and drop in-minibatch from --num-micro-batches-in-minibatch

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments