Commits · 1aa2e08afd9858a42a227527637032a76e29bc8e · OpenDAS / Megatron-LM

20 Jan, 2021 1 commit
- Merge branch 'fix_zeroshot_bug' into 'main' · 1aa2e08a
  Jared Casper authored Jan 20, 2021
```
GPT2->GPT in zero-shot evaluation scripts

See merge request ADLR/megatron-lm!210
```
  1aa2e08a
17 Jan, 2021 1 commit
- GPT2->GPT in zero-shot evaluation scripts · 5ff0f882
  Deepak Narayanan authored Jan 17, 2021
  
  5ff0f882
13 Jan, 2021 2 commits
- Merge branch 'transformer_decoder' into 'main' · f34cc86b
  Jared Casper authored Jan 13, 2021
```
Transformer decoder

See merge request ADLR/megatron-lm!205
```
  f34cc86b
- Adressing more review comments · 4ae54b55
  Vijay Korthikanti authored Jan 12, 2021
  
  4ae54b55
12 Jan, 2021 8 commits
- Adress more review comments · d836d498
  Vijay Korthikanti authored Jan 12, 2021
  
  d836d498
- Merge branch 'TOC-fix' into 'main' · be473a5b
  Jared Casper authored Jan 12, 2021
```
TOC fix

See merge request ADLR/megatron-lm!208
```
  be473a5b
- TOC fix · 31733784
  Jared Casper authored Jan 12, 2021
  
  31733784
- address review comments · 4b3519cb
  Vijay Korthikanti authored Jan 12, 2021
  
  4b3519cb
- Merge branch 'readme_update_jan_2021' into 'main' · 4635bdf7
  Jared Casper authored Jan 11, 2021
```
Readme update + change gpt2 to gpt

See merge request ADLR/megatron-lm!206
```
  4635bdf7
- Readme update + change gpt2 to gpt · 152aab30
  Mohammad Shoeybi authored Jan 11, 2021
  
  152aab30
- Merge branch 'validation_loss_reporting' into 'main' · 86eb5bd8
  Jared Casper authored Jan 11, 2021
```
fixed validation loss reporting in tensorboard

See merge request ADLR/megatron-lm!207
```
  86eb5bd8
- fixed validation loss reporting in tensorboard · 5b74f764
  mohammad authored Jan 11, 2021
  
  5b74f764
09 Jan, 2021 2 commits
- minor fixes · 834d6dd5
  Vijay Korthikanti authored Jan 08, 2021
  
  834d6dd5
- decoder support in transformers · 4b506832
  Vijay Korthikanti authored Jan 08, 2021
  
  4b506832
07 Jan, 2021 2 commits
- Merge branch 'default_args' into 'main' · f5eac3d1
  Mohammad Shoeybi authored Jan 06, 2021
```
Change some arguments to default to on.

See merge request ADLR/megatron-lm!203
```
  f5eac3d1
- Change some arguments to default to on. · 9acc8956
  Jared Casper authored Jan 06, 2021
  
  9acc8956
06 Jan, 2021 4 commits
- Merge branch 'optimize_refactor' into 'main' · 9a297541
  Jared Casper authored Jan 06, 2021
```
Optimizer refactor

See merge request ADLR/megatron-lm!197
```
  9a297541
- Merge branch 'optimize_refactor_backward_compat' into 'optimize_refactor' · 42c6d8e7
  Jared Casper authored Jan 05, 2021
```
added backward compatibility to be able to load checkpoints

See merge request ADLR/megatron-lm!200
```
  42c6d8e7
- fixed assert issue in clip grads for none grads · 82945c1a
  mohammad authored Jan 05, 2021
  
  82945c1a
- added comment about norm power · 571f10a0
  mohammad authored Jan 05, 2021
  
  571f10a0
05 Jan, 2021 13 commits
- addressed Jareds and Deepaks comments · db88a27b
  mohammad authored Jan 05, 2021
  
  db88a27b
- addressed jareds comments · 7417078a
  mohammad authored Jan 05, 2021
  
  7417078a
- Merge branch 'task_head_fix' into 'main' · b69bc7ef
  Mohammad Shoeybi authored Jan 05, 2021
```
Only create task heads on last pipeline stage.

See merge request ADLR/megatron-lm!202
```
  b69bc7ef
- Merge branch 'optimize_refactor_finetune_fix' into 'optimize_refactor' · 512337f5
  Mohammad Shoeybi authored Jan 05, 2021
```
Use new api to get loss scale when finetuning.

See merge request ADLR/megatron-lm!201
```
  512337f5
- Only create task heads on last pipeline stage. · f772fbc9
  Jared Casper authored Jan 05, 2021
  
  f772fbc9
- Use new api to get loss scale when finetuning. · a13cbe1e
  Jared Casper authored Jan 05, 2021
  
  a13cbe1e
- Merge branch 'optimize_refactor_reload_model_params' into 'optimize_refactor' · 7381754e
  Jared Casper authored Jan 05, 2021
```
Optimize refactor reload model params

See merge request ADLR/megatron-lm!199
```
  7381754e
- Merge branch 'send_and_recv' into 'main' · 6fa36844
  Jared Casper authored Jan 05, 2021
```
Use batched send and recv instead of torch.distributed.ring_exchange()

See merge request ADLR/megatron-lm!198
```
  6fa36844
- some small fixes · 9eedf896
  mohammad authored Jan 04, 2021
  
  9eedf896
- added backward compatibility · 203235fd
  mohammad authored Jan 04, 2021
  
  203235fd
- further refactor, tested, and changed master to main · 983cc311
  mohammad authored Jan 04, 2021
  
  983cc311
- further refactor, tested, and changed master to main · 22fa9bac
  mohammad authored Jan 04, 2021
  
  22fa9bac
- added reload model params for finetuning · 160ba680
  mohammad authored Jan 04, 2021
  
  160ba680
04 Jan, 2021 1 commit
- Use batched send and recv instead of torch.distributed.ring_exchange() · d899988e
  Deepak Narayanan authored Jan 04, 2021
  
  d899988e
31 Dec, 2020 1 commit
- addressed deepaks comments · 43529f78
  mohammad authored Dec 30, 2020
  
  43529f78
30 Dec, 2020 5 commits
- moved compile helper to initialize · 242770dd
  mshoeybi authored Dec 29, 2020
  
  242770dd
- added clip grads · a495871b
  mohammad authored Dec 29, 2020
  
  a495871b
- done with refactoring and checks · b77d9062
  mohammad authored Dec 29, 2020
  
  b77d9062
- add multi-tensor-apply to clip grad · 9a010310
  mohammad authored Dec 29, 2020
  
  9a010310
- moved grad scaling to multi-tensor-apply · 345f5d0d
  mohammad authored Dec 29, 2020
  
  345f5d0d