Commits · 601b19b781f5c9338fc992cc16aac182208213e0 · OpenDAS / Megatron-LM

31 Mar, 2020 1 commit
- tasks tested · 601b19b7
  Mohammad authored Mar 30, 2020
  
  601b19b7
30 Mar, 2020 7 commits
- removed unused values in samplers · 259062c2
  Mohammad authored Mar 30, 2020
  
  259062c2
- added samplers from data_utils · 32e1ddb0
  Mohammad authored Mar 30, 2020
  
  32e1ddb0
- removed building tokenizer from bert dataset · dedb2ef7
  Mohammad authored Mar 30, 2020
  
  dedb2ef7
- both bert and gpt2 tested and working · 1788c910
  Mohammad authored Mar 29, 2020
  
  1788c910
- both bert and gpt are working · 5f8623db
  Mohammad authored Mar 29, 2020
  
  5f8623db
- added gpt2 tokenizer · 46379244
  Mohammad authored Mar 29, 2020
  
  46379244
- added gpt2 tokenizer · 0e5dfd7f
  Mohammad authored Mar 29, 2020
  
  0e5dfd7f
29 Mar, 2020 5 commits
- refactored learning-rate · b6e0377b
  Mohammad authored Mar 29, 2020
  
  b6e0377b
- refactored pretrain-bert · 3366a5b0
  Mohammad authored Mar 28, 2020
  
  3366a5b0
- refactored training · 27e14f82
  Mohammad authored Mar 28, 2020
  
  27e14f82
- utils partially refactored and tested · 3f58649b
  Mohammad authored Mar 28, 2020
  
  3f58649b
- refactored checkpooints but not tested yet · 86e7d624
  Mohammad authored Mar 28, 2020
  
  86e7d624
28 Mar, 2020 4 commits
- tokenizer moved to its own directory · 11220df8
  Mohammad authored Mar 28, 2020
  
  11220df8
- working on utils · 5050203f
  Mohammad authored Mar 28, 2020
  
  5050203f
- arguments.py refactored · 94e2ca57
  Mohammad authored Mar 27, 2020
  
  94e2ca57
- added initialize megatron · a9e19f8e
  Mohammad authored Mar 27, 2020
  
  a9e19f8e
27 Mar, 2020 2 commits
- added global variables · 83aa9219
  Mohammad authored Mar 27, 2020
  
  83aa9219
- working on bert · 599e959a
  Mohammad authored Mar 26, 2020
  
  599e959a
26 Mar, 2020 1 commit
- working on args · 1446bb64
  Mohammad authored Mar 26, 2020
  
  1446bb64
24 Mar, 2020 2 commits
- Merge branch 'staging_tasks' into 'staging' · 37ff534f
  Raul Puri authored Mar 23, 2020
```
BERT downstream tasks: RACE, MNLI, QQP

See merge request ADLR/megatron-lm!35
```
  37ff534f
- BERT downstream tasks: RACE, MNLI, QQP · 90ef2e28
  Mohammad Shoeybi authored Mar 23, 2020
  
  90ef2e28
17 Mar, 2020 5 commits
- Merge branch 'staging-gen-fix' into 'staging' · 41141526
  Raul Puri authored Mar 17, 2020
```
Update generate_samples.py

See merge request ADLR/megatron-lm!34
```
  41141526
- Update generate_samples.py · 323e75c4
  Raul Puri authored Mar 17, 2020
  
  323e75c4
- Merge branch 'staging_move_albert' into 'staging' · 29d10a36
  Jared Casper authored Mar 17, 2020
```
Rename albert to bert and removing original bert

See merge request ADLR/megatron-lm!33
```
  29d10a36
- moved albert to bert · 09e05c6f
  Mohammad Shoeybi authored Mar 16, 2020
  
  09e05c6f
- moved pretrain albert to pretrain bert · 3e4e1ab2
  Mohammad Shoeybi authored Mar 16, 2020
  
  3e4e1ab2
11 Mar, 2020 2 commits
- Merge branch 'memory_optimization' into 'master' · 70174ae3
  Raul Puri authored Mar 10, 2020
```
memory optimization in mpu cross entropy

See merge request ADLR/megatron-lm!32
```
  70174ae3
- memory optimization in mpu cross entropy · 57064fd6
  Mohammad Shoeybi authored Mar 10, 2020
  
  57064fd6
10 Feb, 2020 2 commits
- Merge branch 'mp_merger' into 'master' · ca8dd4ac
  Jared Casper authored Feb 10, 2020
```
Model parallel merger

See merge request ADLR/megatron-lm!28
```
  ca8dd4ac
- Model parallel merger · 57c2060f
  Mohammad Shoeybi authored Feb 10, 2020
  
  57c2060f
04 Feb, 2020 3 commits
- Merge branch 'fixfp16gensamples' into 'master' · 5df85022
  Mohammad Shoeybi authored Feb 04, 2020
```
fixed a bug on fp16 while generating samples

See merge request ADLR/megatron-lm!29
```
  5df85022
- fixed a bug on fp16 while generating samples · cfc6924b
  Mostofa Patwary authored Feb 04, 2020
  
  cfc6924b
- Merge branch 'scale_qkt_exp_lr' into 'master' · a4cb4153
  Jared Casper authored Feb 04, 2020
```
Scale Q*K (query times key) by 1/layer-number and add exponential decay option

See merge request ADLR/megatron-lm!27
```
  a4cb4153
17 Jan, 2020 2 commits
- Merge branch 'model_parallel_generation' into 'master' · 4ad5717e
  Raul Puri authored Jan 16, 2020
```
force output gathering

See merge request ADLR/megatron-lm!17
```
  4ad5717e
- undid padding merge · 7a6d630e
  Raul Puri authored Jan 16, 2020
  
  7a6d630e
14 Jan, 2020 3 commits
- added query-key layer scaling and softmax fp32 option · 6c2d0337
  Mohammad Shoeybi authored Jan 13, 2020
  
  6c2d0337
- added query-key layer scaling and softmax fp32 option · 691747b1
  Mohammad Shoeybi authored Jan 13, 2020
  
  691747b1
- exponential learning rate decay added · 7a3b4c15
  Mohammad Shoeybi authored Jan 13, 2020
  
  7a3b4c15
06 Jan, 2020 1 commit
- Merge branch 'mo_albert_data_loader' into 'master' · 2d76d065
  Jared Casper authored Jan 06, 2020
```
New data loader

See merge request ADLR/megatron-lm!16
```
  2d76d065