Commits · 7abd3e90d09cf02a3a019c9a2af67ee31d5b1bd4 · OpenDAS / Megatron-LM

12 Nov, 2020 1 commit

Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90

Deepak Narayanan authored Aug 29, 2020

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

7abd3e90

29 Sep, 2020 1 commit
- Text generation bugs · 1f3f834c
  Mostofa Patwary authored Sep 28, 2020
  
  1f3f834c
16 Apr, 2020 1 commit
- changed licence 2019 to 2020 · 463d1257
  Mohammad authored Apr 16, 2020
  
  463d1257
15 Apr, 2020 1 commit
- changed gpt2 masking to binary and masked_fill · 8ceed7c7
  Mohammad authored Apr 14, 2020
  
  8ceed7c7
14 Apr, 2020 1 commit
- Lint whole repo · b7f1b050
  Neel Kant authored Apr 14, 2020
  
  b7f1b050
03 Apr, 2020 4 commits
- sample generation runs · fffa0497
  Mohammad authored Apr 02, 2020
  
  fffa0497
- code runs · 752eeae3
  Mohammad authored Apr 02, 2020
  
  752eeae3
- generate samples linted · a6ba254f
  Mohammad authored Apr 02, 2020
  
  a6ba254f
- working on refactoring text generation · ce29d4d5
  Mohammad authored Apr 02, 2020
  
  ce29d4d5
29 Mar, 2020 1 commit
- utils partially refactored and tested · 3f58649b
  Mohammad authored Mar 28, 2020
  
  3f58649b
17 Mar, 2020 1 commit
- Update generate_samples.py · 323e75c4
  Raul Puri authored Mar 17, 2020
  
  323e75c4
04 Feb, 2020 1 commit
- fixed a bug on fp16 while generating samples · cfc6924b
  Mostofa Patwary authored Feb 04, 2020
  
  cfc6924b
22 Dec, 2019 1 commit
- force output gathering · b9b6fe0d
  Raul Puri authored Dec 22, 2019
  
  b9b6fe0d
27 Nov, 2019 1 commit
- refactoring · f223ff67
  Mohammad Shoeybi authored Nov 27, 2019
  
  f223ff67
04 Nov, 2019 1 commit
- Fixed edge case with multiple end of sequence in one sequence · 872b4a6a
  Raul Puri authored Nov 04, 2019
  
  872b4a6a
29 Oct, 2019 1 commit
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
08 Oct, 2019 1 commit
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb
04 Oct, 2019 1 commit
- Updating public repo with latest changes. · a1d04b79
  Jared Casper authored Oct 04, 2019
  
  a1d04b79
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e