- 22 Mar, 2021 1 commit
-
-
Jared Casper authored
-
- 13 Feb, 2021 1 commit
-
-
Deepak Narayanan authored
-
- 09 Feb, 2021 2 commits
-
-
Deepak Narayanan authored
-
Deepak Narayanan authored
- Split a model's computation into multiple virtual stages as needed, and schedule communication correctly between these virtual stages - Move schedule code into `schedules.py` and communication code into `p2p_communication.py` - Use hyphens instead of spaces in all time logging for consistency - Factor out code in megatron/training.py into helper functions - Refactor evaluate() function: make it use forward_backward_schedule functions
-
- 26 Jan, 2021 1 commit
-
-
mohammad authored
-
- 25 Jan, 2021 2 commits
-
-
Jared Casper authored
-
Jared Casper authored
-
- 22 Jan, 2021 1 commit
-
-
Jared Casper authored
-
- 08 Jan, 2021 1 commit
-
-
Vijay Korthikanti authored
-
- 04 Jan, 2021 1 commit
-
-
Deepak Narayanan authored
-
- 25 Dec, 2020 3 commits
- 22 Dec, 2020 1 commit
-
-
mohammad authored
Add the option for fp32 residual connection (fp32 residual connection machinery still needs to be added)
-
- 19 Dec, 2020 4 commits
-
-
Jared Casper authored
-
Jared Casper authored
-
mohammad authored
-
mohammad authored
Rename --batch-size to --micro-batch-size and drop in-minibatch from --num-micro-batches-in-minibatch
-
- 18 Nov, 2020 1 commit
-
-
Jared Casper authored
-
- 12 Nov, 2020 3 commits
-
-
Deepak Narayanan authored
-
Deepak Narayanan authored
-
Deepak Narayanan authored
Also includes following changes for inter-layer model-parallel implementation: - Refactoring of model implementations - Training loop changes to support inter-layer communication using `ring_exchange` - New groups for inter-layer communication - Checkpoint changes - Command line arguments
-
- 02 Oct, 2020 1 commit
-
-
Mostofa Patwary authored
-
- 11 Sep, 2020 1 commit
-
-
Vijay Korthikanti authored
-
- 03 Sep, 2020 1 commit
-
-
mohammad authored
-
- 02 Sep, 2020 2 commits
- 27 Aug, 2020 1 commit
-
-
Boris Fomitchev authored
Signed-off-by:Boris Fomitchev <bfomitchev@nvidia.com>
-
- 26 Aug, 2020 1 commit
-
-
Boris Fomitchev authored
-
- 07 Aug, 2020 2 commits
- 06 Aug, 2020 1 commit
-
-
Boris Fomitchev authored
Signed-off-by:Boris Fomitchev <bfomitchev@nvidia.com>
-
- 03 Aug, 2020 1 commit
-
-
mohammad authored
-
- 31 Jul, 2020 1 commit
-
-
Boris Fomitchev authored
Signed-off-by:Boris Fomitchev <bfomitchev@nvidia.com>
-
- 30 Jul, 2020 1 commit
-
-
Boris Fomitchev authored
Signed-off-by:Boris Fomitchev <bfomitchev@nvidia.com>
-
- 26 Jul, 2020 2 commits
-
-
Neel Kant authored
-
Boris Fomitchev authored
-
- 24 Jul, 2020 1 commit
-
-
Boris Fomitchev authored
Signed-off-by:Boris Fomitchev <bfomitchev@nvidia.com>
-
- 04 Jul, 2020 1 commit
-
-
Devendra Singh Sachan authored
-
- 05 Jun, 2020 1 commit
-
-
Neel Kant authored
-