Commits · c1faa9fecff62f6d48b08b7a3596bd4205da196b · OpenDAS / Megatron-LM

08 Feb, 2021 1 commit
- Improve handling of rng states in checkpoints. · 08a848c7
  Jared Casper authored Feb 08, 2021
  
  08a848c7
05 Feb, 2021 2 commits
- Use torch.cuda.synchronize() right after calling batch_isend_irecv() communication API · 7ffea978
  Deepak Narayanan authored Feb 05, 2021
  
  7ffea978
- address review comments · 0cb36de2
  Vijay Korthikanti authored Feb 05, 2021
  
  0cb36de2
04 Feb, 2021 1 commit
- conditioning fused kernels · 4916bae6
  Vijay Korthikanti authored Feb 04, 2021
  
  4916bae6
29 Jan, 2021 1 commit
- added option to change tensorboard queue size · e9b90500
  mohammad authored Jan 29, 2021
  
  e9b90500
28 Jan, 2021 4 commits
- changed validation loss name · 792a468d
  mohammad authored Jan 28, 2021
  
  792a468d
- added options for tensorboard logging · 3a26a168
  mohammad authored Jan 28, 2021
  
  3a26a168
- Typo fix. · e1f574cd
  Jared Casper authored Jan 28, 2021
  
  e1f574cd
- license text for autoaugmentation · 54bb3046
  Vijay Korthikanti authored Jan 28, 2021
  
  54bb3046
27 Jan, 2021 4 commits
- Move rearranging query_key_value and key_value values in old checkpoints to... · 76960d7c
  Jared Casper authored Jan 27, 2021
```
Move rearranging query_key_value and key_value values in old checkpoints to when the checkpoint is loaded instead of runtime..
```
  76960d7c
- Teach merge_mp_partitions how to write out a pipelined model. · 7cabbe67
  Jared Casper authored Jan 27, 2021
  
  7cabbe67
- added flag so we dont calculate params norm all the time · 3dcbaec9
  mohammad authored Jan 26, 2021
  
  3dcbaec9
- vit model does not get imported automatically · ab507293
  mohammad authored Jan 26, 2021
  
  ab507293
26 Jan, 2021 4 commits
- dataloader_type argument fix + randomsampler fix · 85589322
  Vijay Korthikanti authored Jan 25, 2021
  
  85589322
- added params norms · 57d1356e
  mohammad authored Jan 25, 2021
  
  57d1356e
- call makefile every run so we recompile if the code has changed · f0232865
  mohammad authored Jan 25, 2021
  
  f0232865
- Adressing more review comments · e6c7b05e
  Vijay Korthikanti authored Jan 25, 2021
  
  e6c7b05e
25 Jan, 2021 4 commits
- Clarify module.initialize_word_embeddings. · 7be2648a
  Jared Casper authored Jan 25, 2021
  
  7be2648a
- Adding option to remove the binary head for BERT · dcff1acd
  Mohammad Shoeybi authored Jan 25, 2021
  
  dcff1acd
- Use set_tensor_model_parallel_attributes in bert_model as well. · c4c68dce
  Jared Casper authored Jan 25, 2021
  
  c4c68dce
- Address comments, fix argument bug. · bdd47d64
  Jared Casper authored Jan 25, 2021
  
  bdd47d64
24 Jan, 2021 1 commit
- added grad norm to logging and tensorboard · 00764415
  mohammad authored Jan 23, 2021
  
  00764415
22 Jan, 2021 3 commits
- Fixing merge_mp_partitions · 78066ab0
  Jared Casper authored Jan 20, 2021
  
  78066ab0
- Addressing review comments · a7169297
  Vijay Korthikanti authored Jan 22, 2021
  
  a7169297
- attention_mask_func cleanup · ebf8b89e
  Vijay Korthikanti authored Jan 22, 2021
  
  ebf8b89e
13 Jan, 2021 1 commit
- Adressing more review comments · 4ae54b55
  Vijay Korthikanti authored Jan 12, 2021
  
  4ae54b55
12 Jan, 2021 4 commits
- Adress more review comments · d836d498
  Vijay Korthikanti authored Jan 12, 2021
  
  d836d498
- address review comments · 4b3519cb
  Vijay Korthikanti authored Jan 12, 2021
  
  4b3519cb
- Readme update + change gpt2 to gpt · 152aab30
  Mohammad Shoeybi authored Jan 11, 2021
  
  152aab30
- fixed validation loss reporting in tensorboard · 5b74f764
  mohammad authored Jan 11, 2021
  
  5b74f764
09 Jan, 2021 2 commits
- minor fixes · 834d6dd5
  Vijay Korthikanti authored Jan 08, 2021
  
  834d6dd5
- decoder support in transformers · 4b506832
  Vijay Korthikanti authored Jan 08, 2021
  
  4b506832
08 Jan, 2021 2 commits
- minor cleanup · a75f1783
  Vijay Korthikanti authored Jan 08, 2021
  
  a75f1783
- vision transformer model and vision classification task · 456f1728
  Vijay Korthikanti authored Jan 08, 2021
  
  456f1728
07 Jan, 2021 1 commit
- Change some arguments to default to on. · 9acc8956
  Jared Casper authored Jan 06, 2021
  
  9acc8956
06 Jan, 2021 2 commits
- fixed assert issue in clip grads for none grads · 82945c1a
  mohammad authored Jan 05, 2021
  
  82945c1a
- added comment about norm power · 571f10a0
  mohammad authored Jan 05, 2021
  
  571f10a0
05 Jan, 2021 3 commits
- addressed Jareds and Deepaks comments · db88a27b
  mohammad authored Jan 05, 2021
  
  db88a27b
- addressed jareds comments · 7417078a
  mohammad authored Jan 05, 2021
  
  7417078a
- Only create task heads on last pipeline stage. · f772fbc9
  Jared Casper authored Jan 05, 2021
  
  f772fbc9