Commits · f5eac3d19b89991d2e2c88f50efeb0fe5cc69a45 · OpenDAS / Megatron-LM

"docs/source/Contributors_Guide.rst" did not exist on "19490ac4f71735495bcdd080a2531d679c3573e9"

13 Nov, 2020 1 commit
- New example scripts showing how to use MP, and some notes to main README · a5a41922
  Deepak Narayanan authored Nov 13, 2020
  
  a5a41922
12 Nov, 2020 2 commits

Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
Deepak Narayanan authored Oct 20, 2020

52a5f2f2

Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90

Deepak Narayanan authored Aug 29, 2020

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

7abd3e90

16 Apr, 2020 2 commits
- latest · 1f45adcc
  Mohammad authored Apr 16, 2020
  
  1f45adcc
- moved steves branch · 78cb1781
  Mohammad authored Apr 16, 2020
  
  78cb1781
08 Apr, 2020 1 commit
- Rename and organize scripts directory. · 985f027d
  Jared Casper authored Apr 08, 2020
  
  985f027d
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e

13 May, 2019 1 commit
- added presplit-sentences to scripts · 3573423f
  Raul Puri authored May 13, 2019
  
  3573423f
27 Mar, 2019 1 commit
- initial commit · fb4cbdc2
  Raul Puri authored Mar 27, 2019
  
  fb4cbdc2