Commits · ef59b68720a787eebbc7bbf186f333f807aaf02e · OpenDAS / Megatron-LM

17 Feb, 2023 1 commit
- Fix distributed pretraining examples · 94dbfd1c
  Maanu Grover authored Feb 17, 2023
  
  94dbfd1c
19 May, 2022 1 commit
- Update readme with sequence parallelism and selective activation recompute. · b83477a3
  Jared Casper authored May 19, 2022
  
  b83477a3
19 Aug, 2021 1 commit

Checkpoint a set number of invidividual Transformer layers · c1e0689d

slym authored Aug 10, 2021

consider the case of pipeline-model prallelism

clean up arugments

argument naming cleanup

update readme and examples

c1e0689d

12 Jan, 2021 1 commit
- Readme update + change gpt2 to gpt · 152aab30
  Mohammad Shoeybi authored Jan 11, 2021
  
  152aab30
13 Nov, 2020 1 commit
- New example scripts showing how to use MP, and some notes to main README · a5a41922
  Deepak Narayanan authored Nov 13, 2020
  
  a5a41922
12 Nov, 2020 1 commit
- Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
  Deepak Narayanan authored Oct 20, 2020
  
  52a5f2f2
16 Apr, 2020 2 commits
- latest · 1f45adcc
  Mohammad authored Apr 16, 2020
  
  1f45adcc
- moved steves branch · 78cb1781
  Mohammad authored Apr 16, 2020
  
  78cb1781
08 Apr, 2020 1 commit
- Rename and organize scripts directory. · 985f027d
  Jared Casper authored Apr 08, 2020
  
  985f027d
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e