GitLab

Some cleanup of types, etc. in the helpers cpp code.

Merge branch 'albert_data_loader' of ssh://gitlab-master.nvidia.com:12051/ADLR/megatron-lm into albert_data_loader

Also removed the fix_lua_indexing stuff from fairseq.

Fixed edge case with multiple end of sequence in one sequence

See merge request ADLR/megatron-lm!12

refactored for code reuse

See merge request ADLR/megatron-lm!11

Major refactoring, combining gpt2 and bert

See merge request ADLR/megatron-lm!8

Move vocab padding to utils

See merge request ADLR/megatron-lm!6

fixed deserializing issue with old checkpoint

See merge request ADLR/megatron-lm!5

moved few common elements between bert and gpt to utils

See merge request ADLR/megatron-lm!4

created megatron package

See merge request ADLR/megatron-lm!3

Use DDP command line argument instead of source flag in pretrain_bert.py.

See merge request ADLR/megatron-lm!2

Note that there is currently an issue with bert using Torch DDP.

Updating public repo with latest changes.

See merge request ADLR/megatron-lm!1

Fixes #7.

Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>