Commits · b1c4a2d6e9a56cbbcdbdb0e54ba4efa2df5378e1 · OpenDAS / Megatron-LM

08 Apr, 2020 3 commits
- Document indexed_dataset.get() a bit. · b1c4a2d6
  Jared Casper authored Apr 08, 2020
  
  b1c4a2d6
- Updates to preprocess_data.py and indexed_dataset. · da0562fc
  Jared Casper authored Apr 07, 2020
```
preprocess_data:
- Adds ability to not split sentences. This is used for gpt2 datasets.

- Adds ability to create multiple datasets from different json keys,
this is current untested.

indexed_dataset:
- Add new "get" function to get a portion of an entry.
```
  da0562fc
- added build sample index to c++ · f66c58a9
  Mohammad authored Apr 07, 2020
  
  f66c58a9
07 Apr, 2020 1 commit
- added new gpt2 dataloder · fe207787
  Mohammad authored Apr 06, 2020
  
  fe207787
03 Apr, 2020 8 commits
- Merge branch 'dataset-utils-lint' into 'staging' · 1eccd462
  Mohammad Shoeybi authored Apr 03, 2020
```
Lint megatron/data/dataset_utils.py

See merge request ADLR/megatron-lm!42
```
  1eccd462
- Lint megatron/data/dataset_utils.py · 840759b8
  Neel Kant authored Apr 03, 2020
  
  840759b8
- Merge branch 'staging_text_generation' into 'staging' · 63262827
  Raul Puri authored Apr 03, 2020
```
Refactoring text generation

See merge request ADLR/megatron-lm!39
```
  63262827
- sample generation runs · fffa0497
  Mohammad authored Apr 02, 2020
  
  fffa0497
- code runs · 752eeae3
  Mohammad authored Apr 02, 2020
  
  752eeae3
- generate samples linted · a6ba254f
  Mohammad authored Apr 02, 2020
  
  a6ba254f
- Merge branch 'staging' into staging_text_generation · a19820b1
  Mohammad authored Apr 02, 2020
  
  a19820b1
- working on refactoring text generation · ce29d4d5
  Mohammad authored Apr 02, 2020
  
  ce29d4d5
02 Apr, 2020 8 commits
- Merge branch 'staging_gpt2_evals' into 'staging' · 3977b721
  Raul Puri authored Apr 02, 2020
```
Refactoring evaluate gpt2

See merge request ADLR/megatron-lm!38
```
  3977b721
- Merge branch 'staging' into staging_gpt2_evals · a0bcee94
  Mohammad authored Apr 02, 2020
  
  a0bcee94
- some changes to args · 2bd6196c
  Mohammad authored Apr 02, 2020
  
  2bd6196c
- verified zeroshot tasks works · ba2264ab
  Raul Puri authored Apr 02, 2020
  
  ba2264ab
- Merge branch 'staging_args' into 'staging' · 6c521eb1
  Jared Casper authored Apr 02, 2020
```
Major refactor: args, global variables, tokenizer

See merge request ADLR/megatron-lm!36
```
  6c521eb1
- Add linting script · b07f1d0a
  Neel Kant authored Apr 01, 2020
  
  b07f1d0a
- gpt2 zeroshots added to main · 159fcfeb
  Mohammad authored Apr 01, 2020
  
  159fcfeb
- refactored evaluate_gpt2 · 8fae1cdd
  Mohammad authored Apr 01, 2020
  
  8fae1cdd
01 Apr, 2020 1 commit
- addressed comments from raul, neel, and jared · bf3ce751
  Mohammad authored Apr 01, 2020
  
  bf3ce751
31 Mar, 2020 3 commits
- deprecated data_utils · 8600642e
  Mohammad authored Mar 31, 2020
  
  8600642e
- arguments in the model refactored · deffcb6a
  Mohammad authored Mar 31, 2020
  
  deffcb6a
- tasks tested · 601b19b7
  Mohammad authored Mar 30, 2020
  
  601b19b7
30 Mar, 2020 7 commits
- removed unused values in samplers · 259062c2
  Mohammad authored Mar 30, 2020
  
  259062c2
- added samplers from data_utils · 32e1ddb0
  Mohammad authored Mar 30, 2020
  
  32e1ddb0
- removed building tokenizer from bert dataset · dedb2ef7
  Mohammad authored Mar 30, 2020
  
  dedb2ef7
- both bert and gpt2 tested and working · 1788c910
  Mohammad authored Mar 29, 2020
  
  1788c910
- both bert and gpt are working · 5f8623db
  Mohammad authored Mar 29, 2020
  
  5f8623db
- added gpt2 tokenizer · 46379244
  Mohammad authored Mar 29, 2020
  
  46379244
- added gpt2 tokenizer · 0e5dfd7f
  Mohammad authored Mar 29, 2020
  
  0e5dfd7f
29 Mar, 2020 5 commits
- refactored learning-rate · b6e0377b
  Mohammad authored Mar 29, 2020
  
  b6e0377b
- refactored pretrain-bert · 3366a5b0
  Mohammad authored Mar 28, 2020
  
  3366a5b0
- refactored training · 27e14f82
  Mohammad authored Mar 28, 2020
  
  27e14f82
- utils partially refactored and tested · 3f58649b
  Mohammad authored Mar 28, 2020
  
  3f58649b
- refactored checkpooints but not tested yet · 86e7d624
  Mohammad authored Mar 28, 2020
  
  86e7d624
28 Mar, 2020 4 commits
- tokenizer moved to its own directory · 11220df8
  Mohammad authored Mar 28, 2020
  
  11220df8
- working on utils · 5050203f
  Mohammad authored Mar 28, 2020
  
  5050203f
- arguments.py refactored · 94e2ca57
  Mohammad authored Mar 27, 2020
  
  94e2ca57
- added initialize megatron · a9e19f8e
  Mohammad authored Mar 27, 2020
  
  a9e19f8e