GitLab

The LICENSE file says everything is 3-clause BSD, which is what we want,
but at some point the Apache license was added to the top of some files
and that proliferated. This commit removes the Apache license from any
files that we own the copyright to.

Also updates the copyright year and removes the unnessary coding=utf-8
line.

tools/merge_datasets.py
- tool to merge multiple dataset files into a single dataset
- testing conducted and included in the megatron-testing repo https://gitlab-master.nvidia.com/ADLR/megatron-testing

tools/preprocess_data.py
- magic numbers changed to required command line arguments

megatron/data/indexed_dataset.py
- when merging, fix to properly update document index
- testing conducted and included in the megatron-testing repo (see above)
- fix follows this history https://github.com/bigscience-workshop/Megatron-DeepSpeed/pull/66

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

preprocess_data:
- Adds ability to not split sentences. This is used for gpt2 datasets.

- Adds ability to create multiple datasets from different json keys,
this is current untested.

indexed_dataset:
- Add new "get" function to get a portion of an entry.