Commits · 73af1290390a33c22ab1a78de7b0cb05b555aa47 · OpenDAS / Megatron-LM

29 Oct, 2019 1 commit
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
09 Oct, 2019 5 commits
- Merge branch 'move_vocab_padding_to_utils' into 'master' · ebbe40cd
  Raul Puri authored Oct 09, 2019
```
Move vocab padding to utils

See merge request ADLR/megatron-lm!6
```
  ebbe40cd
- Merge branch 'master' into move_vocab_padding_to_utils · 7e46ca58
  Mohammad Shoeybi authored Oct 09, 2019
  
  7e46ca58
- moved padding to utils · 2e6d5ed9
  Mohammad Shoeybi authored Oct 09, 2019
  
  2e6d5ed9
- Merge branch 'load_checkpoint_fix' into 'master' · 6b68bb8a
  Raul Puri authored Oct 09, 2019
```
fixed deserializing issue with old checkpoint

See merge request ADLR/megatron-lm!5
```
  6b68bb8a
- fixed deserializing issue with old checkpoint · ee38e7f9
  Mohammad Shoeybi authored Oct 09, 2019
  
  ee38e7f9
08 Oct, 2019 6 commits
- Merge branch 'refactor_utils' into 'master' · 9993ea25
  Jared Casper authored Oct 08, 2019
```
moved few common elements between bert and gpt to utils

See merge request ADLR/megatron-lm!4
```
  9993ea25
- moved few common elements between bert and gpt to utils · 787c1a0b
  Mohammad Shoeybi authored Oct 08, 2019
  
  787c1a0b
- Merge branch 'megatron_package' into 'master' · c882ac61
  Jared Casper authored Oct 08, 2019
```
created megatron package

See merge request ADLR/megatron-lm!3
```
  c882ac61
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb
- Merge branch 'torchddp' into 'master' · 4947002d
  Mohammad Shoeybi authored Oct 07, 2019
```
Use DDP command line argument instead of source flag in pretrain_bert.py.

See merge request ADLR/megatron-lm!2
```
  4947002d
- Use DDP command line argument instead of source flag in pretrain_bert.py. · 72c5f666
  Jared Casper authored Oct 07, 2019
```
Note that there is currently an issue with bert using Torch DDP.
```
  72c5f666
07 Oct, 2019 2 commits
- Merge branch 'staging' into 'master' · a54978bb
  Jared Casper authored Oct 07, 2019
```
Updating public repo with latest changes.

See merge request ADLR/megatron-lm!1
```
  a54978bb
- Add licence to split script. · 5d402eb4
  Jared Casper authored Oct 07, 2019
  
  5d402eb4
04 Oct, 2019 1 commit
- Updating public repo with latest changes. · a1d04b79
  Jared Casper authored Oct 04, 2019
  
  a1d04b79
12 Sep, 2019 2 commits
- added missing valid-data line (#9) · 93ab4bea
  Raul Puri authored Sep 12, 2019
  
  93ab4bea
- Support latest PyTorch RNG state API. (#8) · 2ef23675
  Jared Casper authored Sep 11, 2019
```
Fixes #7.
```
  2ef23675
13 Aug, 2019 1 commit
- eval+numeric update · a0368ddf
  Raul Puri authored Aug 12, 2019
  
  a0368ddf
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e

14 May, 2019 1 commit
- fixed save race condition · 0399d32c
  Raul Puri authored May 14, 2019
  
  0399d32c
13 May, 2019 1 commit
- added presplit-sentences to scripts · 3573423f
  Raul Puri authored May 13, 2019
  
  3573423f
11 May, 2019 1 commit
- added missing presplit_sentences_json.py · d0878333
  Raul Puri authored May 11, 2019
  
  d0878333
23 Apr, 2019 1 commit

Faster dataloader merge () · 66719e97

Raul Puri authored Apr 23, 2019

* threaded tf_dl+presplit sentences+shuffled dataset with resume

* elaborate in readme

66719e97

27 Mar, 2019 1 commit
- initial commit · fb4cbdc2
  Raul Puri authored Mar 27, 2019
  
  fb4cbdc2