Commits · 61184a8fa52035cbc6bfcdd89a48deda22dd5e15 · OpenDAS / Megatron-LM

02 Jul, 2021 1 commit
- Got 530 Billion parameter model working! · ac3db159
  rprenger authored Jul 02, 2021
  
  ac3db159
30 Jun, 2021 3 commits
- Removing unnecessary --recompute path · 453414da
  rprenger authored Jun 30, 2021
  
  453414da
- Clean up removing a lot of code. Works with curl people might still want a webpage or CLI · f7fe3865
  rprenger authored Jun 30, 2021
  
  f7fe3865
- Simpler broadcasting and some clean up · a9a3ef50
  rprenger authored Jun 30, 2021
  
  a9a3ef50
23 Jun, 2021 1 commit
- Working single sentence version · 2cdebf4b
  rprenger authored Jun 23, 2021
  
  2cdebf4b
21 Apr, 2021 3 commits
- addressed comments · e5ec27d7
  Mostofa Patwary authored Apr 21, 2021
  
  e5ec27d7
- addressed comments · 5a6431f5
  Mostofa Patwary authored Apr 21, 2021
  
  5a6431f5
- addressed comments · 5c2ce593
  Mostofa Patwary authored Apr 21, 2021
  
  5c2ce593
19 Apr, 2021 1 commit
- modified the params · 44bfcb32
  Mostofa Patwary authored Apr 19, 2021
  
  44bfcb32
16 Apr, 2021 1 commit
- added this function for evaluation · 045959cb
  Mostofa Patwary authored Apr 16, 2021
  
  045959cb
24 Mar, 2021 2 commits
- Fixing text generation and zeroshot eval and addressing comments. · 64a83fb5
  Jared Casper authored Mar 24, 2021
  
  64a83fb5
- Tasks seems to be working. · b938ec51
  Jared Casper authored Mar 23, 2021
  
  b938ec51
19 Dec, 2020 3 commits
- Fix text generation without recompute · 1d4e8760
  Jared Casper authored Dec 10, 2020
  
  1d4e8760
- Work batch-size name changes into task code · 3afcba6e
  Jared Casper authored Dec 09, 2020
  
  3afcba6e
- Initial implementation of pipelined text generation · 5c45db4a
  Jared Casper authored Dec 09, 2020
  
  5c45db4a
12 Nov, 2020 2 commits

Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
Deepak Narayanan authored Oct 20, 2020

52a5f2f2

Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90

Deepak Narayanan authored Aug 29, 2020

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

7abd3e90

29 Sep, 2020 1 commit
- Text generation bugs · 1f3f834c
  Mostofa Patwary authored Sep 28, 2020
  
  1f3f834c
16 Apr, 2020 1 commit
- changed licence 2019 to 2020 · 463d1257
  Mohammad authored Apr 16, 2020
  
  463d1257
15 Apr, 2020 1 commit
- changed gpt2 masking to binary and masked_fill · 8ceed7c7
  Mohammad authored Apr 14, 2020
  
  8ceed7c7
14 Apr, 2020 1 commit
- Lint whole repo · b7f1b050
  Neel Kant authored Apr 14, 2020
  
  b7f1b050
03 Apr, 2020 4 commits
- sample generation runs · fffa0497
  Mohammad authored Apr 02, 2020
  
  fffa0497
- code runs · 752eeae3
  Mohammad authored Apr 02, 2020
  
  752eeae3
- generate samples linted · a6ba254f
  Mohammad authored Apr 02, 2020
  
  a6ba254f
- working on refactoring text generation · ce29d4d5
  Mohammad authored Apr 02, 2020
  
  ce29d4d5
29 Mar, 2020 1 commit
- utils partially refactored and tested · 3f58649b
  Mohammad authored Mar 28, 2020
  
  3f58649b
17 Mar, 2020 1 commit
- Update generate_samples.py · 323e75c4
  Raul Puri authored Mar 17, 2020
  
  323e75c4
04 Feb, 2020 1 commit
- fixed a bug on fp16 while generating samples · cfc6924b
  Mostofa Patwary authored Feb 04, 2020
  
  cfc6924b
22 Dec, 2019 1 commit
- force output gathering · b9b6fe0d
  Raul Puri authored Dec 22, 2019
  
  b9b6fe0d
27 Nov, 2019 1 commit
- refactoring · f223ff67
  Mohammad Shoeybi authored Nov 27, 2019
  
  f223ff67
04 Nov, 2019 1 commit
- Fixed edge case with multiple end of sequence in one sequence · 872b4a6a
  Raul Puri authored Nov 04, 2019
  
  872b4a6a
29 Oct, 2019 1 commit
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
08 Oct, 2019 1 commit
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb
04 Oct, 2019 1 commit
- Updating public repo with latest changes. · a1d04b79
  Jared Casper authored Oct 04, 2019
  
  a1d04b79
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e