Commits · 36c2674c19f47882eb03271a8ef305541dd92ce2 · OpenDAS / Megatron-LM

28 Jan, 2021 6 commits
- Merge branch 'ckpt_merge' into 'main' · 36c2674c
  Mohammad Shoeybi authored Jan 28, 2021
```
Teach merge_mp_partitions how to write out a pipelined model.

See merge request ADLR/megatron-lm!218
```
  36c2674c
- Improve comments around layer regex replacement. · 98a5b9a0
  Jared Casper authored Jan 28, 2021
  
  98a5b9a0
- Merge branch 'vision_transformer' into 'main' · de722164
  Jared Casper authored Jan 28, 2021
```
license text for autoaugmentation

See merge request ADLR/megatron-lm!220
```
  de722164
- change to LICENSE file · dcea434a
  Vijay Korthikanti authored Jan 28, 2021
  
  dcea434a
- Merge branch 'ckpt_transpose' into 'main' · 4468f3e4
  Jared Casper authored Jan 28, 2021
```
Rework handling of older checkpoint's attention weight/bias ordering.

See merge request ADLR/megatron-lm!219
```
  4468f3e4
- license text for autoaugmentation · 54bb3046
  Vijay Korthikanti authored Jan 28, 2021
  
  54bb3046
27 Jan, 2021 10 commits
- Move rearranging query_key_value and key_value values in old checkpoints to... · 76960d7c
  Jared Casper authored Jan 27, 2021
```
Move rearranging query_key_value and key_value values in old checkpoints to when the checkpoint is loaded instead of runtime..
```
  76960d7c
- Teach merge_mp_partitions how to write out a pipelined model. · 7cabbe67
  Jared Casper authored Jan 27, 2021
  
  7cabbe67
- Merge branch 'gpt3_script_fix' into 'main' · c7444380
  Jared Casper authored Jan 27, 2021
```
added init method std to gpt3 example

See merge request ADLR/megatron-lm!217
```
  c7444380
- added init method std · a6bf1a04
  mshoeybi authored Jan 26, 2021
  
  a6bf1a04
- Merge branch 'log_grad_norm' into 'main' · 60704e72
  Jared Casper authored Jan 26, 2021
```
added grad and params norm to logging and tensorboard

See merge request ADLR/megatron-lm!214
```
  60704e72
- added flag so we dont calculate params norm all the time · 3dcbaec9
  mohammad authored Jan 26, 2021
  
  3dcbaec9
- Merge branch 'main' into log_grad_norm · 929c780c
  mohammad authored Jan 26, 2021
  
  929c780c
- Merge branch 'vit_einops_fix' into 'main' · 577ad7d3
  Jared Casper authored Jan 26, 2021
```
vit model does not get imported automatically anymore

See merge request ADLR/megatron-lm!216
```
  577ad7d3
- vit model does not get imported automatically · ab507293
  mohammad authored Jan 26, 2021
  
  ab507293
- merge conflict resolved · 286c4d97
  mohammad authored Jan 26, 2021
  
  286c4d97
26 Jan, 2021 9 commits
- Merge branch 'helpers_compilation' into 'main' · 1a92ce5c
  Jared Casper authored Jan 26, 2021
```
call helpers makefile every run so we recompile if the code has changed

See merge request ADLR/megatron-lm!215
```
  1a92ce5c
- Merge branch 'vision_transformer' into 'main' · 996aca68
  Jared Casper authored Jan 26, 2021
```
Vision transformer

See merge request ADLR/megatron-lm!204
```
  996aca68
- dataloader_type argument fix + randomsampler fix · 85589322
  Vijay Korthikanti authored Jan 25, 2021
  
  85589322
- added params norms · 57d1356e
  mohammad authored Jan 25, 2021
  
  57d1356e
- Merge branch 'main' into log_grad_norm · d31d7e49
  mohammad authored Jan 25, 2021
  
  d31d7e49
- call makefile every run so we recompile if the code has changed · f0232865
  mohammad authored Jan 25, 2021
  
  f0232865
- Adressing more review comments · e6c7b05e
  Vijay Korthikanti authored Jan 25, 2021
  
  e6c7b05e
- Merge branch 'main' into vision_transformer · da8dccd6
  Vijay Korthikanti authored Jan 25, 2021
  
  da8dccd6
- Merge branch 'attention_mask_refactor' into 'main' · c0400306
  Jared Casper authored Jan 25, 2021
```
attention_mask_func cleanup

See merge request ADLR/megatron-lm!212
```
  c0400306
25 Jan, 2021 6 commits
- Merge branch 'ckpt_merge' into 'main' · 7db9c1e5
  Jared Casper authored Jan 25, 2021
```
Fixing merge_mp_partitions

See merge request ADLR/megatron-lm!213
```
  7db9c1e5
- Clarify module.initialize_word_embeddings. · 7be2648a
  Jared Casper authored Jan 25, 2021
  
  7be2648a
- Merge branch 'bert_not_remove_single_sent_option' into 'main' · 6afaffa5
  Jared Casper authored Jan 25, 2021
```
Adding option to remove the binary head for BERT

See merge request ADLR/megatron-lm!211
```
  6afaffa5
- Adding option to remove the binary head for BERT · dcff1acd
  Mohammad Shoeybi authored Jan 25, 2021
  
  dcff1acd
- Use set_tensor_model_parallel_attributes in bert_model as well. · c4c68dce
  Jared Casper authored Jan 25, 2021
  
  c4c68dce
- Address comments, fix argument bug. · bdd47d64
  Jared Casper authored Jan 25, 2021
  
  bdd47d64
24 Jan, 2021 1 commit
- added grad norm to logging and tensorboard · 00764415
  mohammad authored Jan 23, 2021
  
  00764415
22 Jan, 2021 3 commits
- Fixing merge_mp_partitions · 78066ab0
  Jared Casper authored Jan 20, 2021
  
  78066ab0
- Addressing review comments · a7169297
  Vijay Korthikanti authored Jan 22, 2021
  
  a7169297
- attention_mask_func cleanup · ebf8b89e
  Vijay Korthikanti authored Jan 22, 2021
  
  ebf8b89e
20 Jan, 2021 1 commit
- Merge branch 'fix_zeroshot_bug' into 'main' · 1aa2e08a
  Jared Casper authored Jan 20, 2021
```
GPT2->GPT in zero-shot evaluation scripts

See merge request ADLR/megatron-lm!210
```
  1aa2e08a
17 Jan, 2021 1 commit
- GPT2->GPT in zero-shot evaluation scripts · 5ff0f882
  Deepak Narayanan authored Jan 17, 2021
  
  5ff0f882
14 Jan, 2021 1 commit
- Merge branch 'main' into vision_transformer · 58edb19a
  Vijay Korthikanti authored Jan 14, 2021
  
  58edb19a
13 Jan, 2021 2 commits
- Merge branch 'transformer_decoder' into 'main' · f34cc86b
  Jared Casper authored Jan 13, 2021
```
Transformer decoder

See merge request ADLR/megatron-lm!205
```
  f34cc86b
- Adressing more review comments · 4ae54b55
  Vijay Korthikanti authored Jan 12, 2021
  
  4ae54b55