Commits · 9dc3c42a84aa656f583703cf8b6b4f79f712b796 · OpenDAS / Megatron-LM

24 May, 2022 1 commit
- preallocating global buffer to avoid memory fragmentation · 9dc3c42a
  Vijay Korthikanti authored May 24, 2022
  
  9dc3c42a
24 Nov, 2021 1 commit
- Add support for signal-based dynamic checkpointing · dbe6c728
  Szymon Migacz authored Nov 23, 2021
  
  dbe6c728
18 Nov, 2021 1 commit
- Revert "Update __init__.py" · 8f241a96
  Satpal Singh Rathore authored Nov 18, 2021
```
This reverts commit 2b699d44
```
  8f241a96
17 Nov, 2021 1 commit
- Update __init__.py · 2b699d44
  Satpal Singh Rathore authored Nov 17, 2021
```
Removed redundant if-else.
```
  2b699d44
29 Sep, 2021 1 commit
- Remove outdated packaging files. · 6c00f448
  Jared Casper authored Sep 29, 2021
  
  6c00f448
19 Dec, 2020 3 commits
- Fix some bugs, add exit-duration capability · a31833ce
  mshoeybi authored Dec 11, 2020
  
  a31833ce
- Add pipelining to GLUE and RACE tasks · caa9dca5
  Jared Casper authored Nov 30, 2020
  
  caa9dca5
- Add micro-batch size calculator · 6ea23928
  mohammad authored Dec 06, 2020
  
  6ea23928
12 Nov, 2020 1 commit

Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90

Deepak Narayanan authored Aug 29, 2020

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

7abd3e90

06 Aug, 2020 1 commit
- Removed assert in case of external ddp, exported initialize_megatron() · be842037
  Boris Fomitchev authored Aug 06, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  be842037
24 Jun, 2020 1 commit
- More ict_merge changes and interactive testing · 3354081f
  Neel Kant authored Jun 23, 2020
  
  3354081f
14 May, 2020 4 commits
- update · c60fb37e
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  c60fb37e
- helpers.cpp suupport added, version changed, clean up · dd483295
  Evelina Bakhturina authored Apr 28, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  dd483295
- review feedback · cd3529b9
  Evelina Bakhturina authored Apr 21, 2020
  
  cd3529b9
- pip support added · bf1daf86
  Evelina Bakhturina authored Apr 21, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  bf1daf86
16 Apr, 2020 1 commit
- changed licence 2019 to 2020 · 463d1257
  Mohammad authored Apr 16, 2020
  
  463d1257
29 Mar, 2020 1 commit
- utils partially refactored and tested · 3f58649b
  Mohammad authored Mar 28, 2020
  
  3f58649b
29 Oct, 2019 1 commit
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
08 Oct, 2019 1 commit
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e

27 Mar, 2019 1 commit
- initial commit · fb4cbdc2
  Raul Puri authored Mar 27, 2019
  
  fb4cbdc2