Commits · e919dd8e0f192dd58b9b34ef7cc78c6de709c59b · OpenDAS / Megatron-LM

"official/benchmark/keras_cifar_benchmark.py" did not exist on "ae655521533ee1c35ae4d8ac18a7e99e35e3012e"

12 Nov, 2020 3 commits
- Small notes in comments in response to Jared's comments · aa9cae27
  Deepak Narayanan authored Oct 28, 2020
  
  aa9cae27
- Address Jared's comments in README and loss_scaler.py · dd079406
  Deepak Narayanan authored Oct 27, 2020
  
  dd079406
- Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90
  Deepak Narayanan authored Aug 29, 2020
```
Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments
```
  7abd3e90
16 Apr, 2020 1 commit
- changed licence 2019 to 2020 · 463d1257
  Mohammad authored Apr 16, 2020
  
  463d1257
15 Apr, 2020 2 commits
- addressed jareds comments · b6e73188
  Mohammad authored Apr 15, 2020
  
  b6e73188
- added multi-tensor-apply to fp16 · 99410264
  Mohammad authored Apr 14, 2020
  
  99410264
14 Apr, 2020 1 commit
- Lint whole repo · b7f1b050
  Neel Kant authored Apr 14, 2020
  
  b7f1b050
08 Oct, 2019 1 commit
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e

27 Mar, 2019 1 commit
- initial commit · fb4cbdc2
  Raul Puri authored Mar 27, 2019
  
  fb4cbdc2