- 12 Nov, 2020 25 commits
-
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
Prevents data_loader from running out of training examples
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
Make sure all forward and backward operations are accounted for
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
Also includes following changes for inter-layer model-parallel implementation: - Refactoring of model implementations - Training loop changes to support inter-layer communication using `ring_exchange` - New groups for inter-layer communication - Checkpoint changes - Command line arguments
-
mohammad authored
-
- 03 Nov, 2020 2 commits
-
-
Deepak Narayanan authored
fixed loss average when all but one value is skipped See merge request ADLR/megatron-lm!164
-
mohammad authored
-
- 14 Oct, 2020 6 commits
-
-
Mohammad Shoeybi authored
fixed wiki evaluation See merge request ADLR/megatron-lm!157
-
Mostofa Patwary authored
-
Mostofa Patwary authored
-
Mohammad Shoeybi authored
Adam betas and eps See merge request ADLR/megatron-lm!156
-
Mostofa Patwary authored
-
Mostofa Patwary authored
-
- 13 Oct, 2020 5 commits
-
-
Mohammad Shoeybi authored
suppress kernel loading output See merge request ADLR/megatron-lm!155
-
Vijay Korthikanti authored
-
Mohammad Shoeybi authored
reordering perf fix See merge request ADLR/megatron-lm!154
-
Vijay Korthikanti authored
-
Vijay Korthikanti authored
-
- 12 Oct, 2020 2 commits
-
-
Vijay Korthikanti authored
-
Vijay Korthikanti authored
-