Commits · 6140718fc3d9a3c1a8ca89edf8ce40ff237671c3 · OpenDAS / Megatron-LM

08 Nov, 2019 5 commits
- before optimization · 6140718f
  Mohammad Shoeybi authored Nov 08, 2019
  
  6140718f
- built simple test for dataset · c125d247
  Mohammad Shoeybi authored Nov 07, 2019
  
  c125d247
- added dataset · 7120e931
  Mohammad Shoeybi authored Nov 07, 2019
  
  7120e931
- added training sample builder · adec01d0
  Mohammad Shoeybi authored Nov 07, 2019
  
  adec01d0
- Add document index to index file. An empty sentence no longer separate documents. · 87bbe9be
  Jared Casper authored Nov 07, 2019
  
  87bbe9be
07 Nov, 2019 2 commits
- Initial commit of multiprocess preprocess and extracted copy of fairseq's indexed_dataset. · 1237533e
  Jared Casper authored Nov 07, 2019
  
  1237533e
- added bert tokenization · 0ceeb3b4
  Mohammad Shoeybi authored Nov 06, 2019
  
  0ceeb3b4
04 Nov, 2019 2 commits
- Merge branch 'multiple-eos-edge-case' into 'master' · 0f873f97
  Mohammad Shoeybi authored Nov 04, 2019
```
Fixed edge case with multiple end of sequence in one sequence

See merge request ADLR/megatron-lm!12
```
  0f873f97
- Fixed edge case with multiple end of sequence in one sequence · 872b4a6a
  Raul Puri authored Nov 04, 2019
  
  872b4a6a
01 Nov, 2019 1 commit
- Merge branch 'code_reuse' into 'master' · cf2f4d9d
  Jared Casper authored Oct 31, 2019
```
refactored for code reuse

See merge request ADLR/megatron-lm!11
```
  cf2f4d9d
31 Oct, 2019 2 commits
- refactored for code reuse · cbd8c054
  Mohammad Shoeybi authored Oct 30, 2019
  
  cbd8c054
- refacotred for code reuse · 34be7dd3
  Mohammad Shoeybi authored Oct 30, 2019
  
  34be7dd3
29 Oct, 2019 2 commits
- Merge branch 'transformer_refactoring_from_pretrain_refactoring' into 'master' · beb3e0d3
  Raul Puri authored Oct 29, 2019
```
Major refactoring, combining gpt2 and bert

See merge request ADLR/megatron-lm!8
```
  beb3e0d3
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
09 Oct, 2019 5 commits
- Merge branch 'move_vocab_padding_to_utils' into 'master' · ebbe40cd
  Raul Puri authored Oct 09, 2019
```
Move vocab padding to utils

See merge request ADLR/megatron-lm!6
```
  ebbe40cd
- Merge branch 'master' into move_vocab_padding_to_utils · 7e46ca58
  Mohammad Shoeybi authored Oct 09, 2019
  
  7e46ca58
- moved padding to utils · 2e6d5ed9
  Mohammad Shoeybi authored Oct 09, 2019
  
  2e6d5ed9
- Merge branch 'load_checkpoint_fix' into 'master' · 6b68bb8a
  Raul Puri authored Oct 09, 2019
```
fixed deserializing issue with old checkpoint

See merge request ADLR/megatron-lm!5
```
  6b68bb8a
- fixed deserializing issue with old checkpoint · ee38e7f9
  Mohammad Shoeybi authored Oct 09, 2019
  
  ee38e7f9
08 Oct, 2019 6 commits
- Merge branch 'refactor_utils' into 'master' · 9993ea25
  Jared Casper authored Oct 08, 2019
```
moved few common elements between bert and gpt to utils

See merge request ADLR/megatron-lm!4
```
  9993ea25
- moved few common elements between bert and gpt to utils · 787c1a0b
  Mohammad Shoeybi authored Oct 08, 2019
  
  787c1a0b
- Merge branch 'megatron_package' into 'master' · c882ac61
  Jared Casper authored Oct 08, 2019
```
created megatron package

See merge request ADLR/megatron-lm!3
```
  c882ac61
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb
- Merge branch 'torchddp' into 'master' · 4947002d
  Mohammad Shoeybi authored Oct 07, 2019
```
Use DDP command line argument instead of source flag in pretrain_bert.py.

See merge request ADLR/megatron-lm!2
```
  4947002d
- Use DDP command line argument instead of source flag in pretrain_bert.py. · 72c5f666
  Jared Casper authored Oct 07, 2019
```
Note that there is currently an issue with bert using Torch DDP.
```
  72c5f666
07 Oct, 2019 2 commits
- Merge branch 'staging' into 'master' · a54978bb
  Jared Casper authored Oct 07, 2019
```
Updating public repo with latest changes.

See merge request ADLR/megatron-lm!1
```
  a54978bb
- Add licence to split script. · 5d402eb4
  Jared Casper authored Oct 07, 2019
  
  5d402eb4
04 Oct, 2019 1 commit
- Updating public repo with latest changes. · a1d04b79
  Jared Casper authored Oct 04, 2019
  
  a1d04b79
12 Sep, 2019 2 commits
- added missing valid-data line (#9) · 93ab4bea
  Raul Puri authored Sep 12, 2019
  
  93ab4bea
- Support latest PyTorch RNG state API. (#8) · 2ef23675
  Jared Casper authored Sep 11, 2019
```
Fixes #7.
```
  2ef23675
13 Aug, 2019 1 commit
- eval+numeric update · a0368ddf
  Raul Puri authored Aug 12, 2019
  
  a0368ddf
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e

14 May, 2019 1 commit
- fixed save race condition · 0399d32c
  Raul Puri authored May 14, 2019
  
  0399d32c
13 May, 2019 1 commit
- added presplit-sentences to scripts · 3573423f
  Raul Puri authored May 13, 2019
  
  3573423f
11 May, 2019 1 commit
- added missing presplit_sentences_json.py · d0878333
  Raul Puri authored May 11, 2019
  
  d0878333
23 Apr, 2019 1 commit

Faster dataloader merge () · 66719e97

Raul Puri authored Apr 23, 2019

* threaded tf_dl+presplit sentences+shuffled dataset with resume

* elaborate in readme

66719e97

27 Mar, 2019 1 commit
- initial commit · fb4cbdc2
  Raul Puri authored Mar 27, 2019
  
  fb4cbdc2