Commits · 78a3dc323f9da3c4f02bcbcafc7d4b06d99ed26c · OpenDAS / Megatron-LM

"next_docs/en/vscode:/vscode.git/clone" did not exist on "aa3df5ffd1a999fdcf7b25bb4c50c43bd63abbf1"

03 Feb, 2021 2 commits
- fixed arguments · 78a3dc32
  Mostofa Patwary authored Feb 03, 2021
  
  78a3dc32
- fixed alignment · a96ad63f
  Mostofa Patwary authored Feb 03, 2021
  
  a96ad63f
01 Feb, 2021 1 commit
- cleared the commented codes · e919dd8e
  Mostofa Patwary authored Feb 01, 2021
  
  e919dd8e
29 Jan, 2021 2 commits
- WIP: main_retriver_merge · 17d897e0
  Mostofa Patwary authored Jan 29, 2021
  
  17d897e0
- added option to change tensorboard queue size · e9b90500
  mohammad authored Jan 29, 2021
  
  e9b90500
28 Jan, 2021 1 commit
- added options for tensorboard logging · 3a26a168
  mohammad authored Jan 28, 2021
  
  3a26a168
27 Jan, 2021 1 commit
- added flag so we dont calculate params norm all the time · 3dcbaec9
  mohammad authored Jan 26, 2021
  
  3dcbaec9
26 Jan, 2021 1 commit
- dataloader_type argument fix + randomsampler fix · 85589322
  Vijay Korthikanti authored Jan 25, 2021
  
  85589322
25 Jan, 2021 2 commits
- Adding option to remove the binary head for BERT · dcff1acd
  Mohammad Shoeybi authored Jan 25, 2021
  
  dcff1acd
- Address comments, fix argument bug. · bdd47d64
  Jared Casper authored Jan 25, 2021
  
  bdd47d64
22 Jan, 2021 2 commits
- Fixing merge_mp_partitions · 78066ab0
  Jared Casper authored Jan 20, 2021
  
  78066ab0
- Addressing review comments · a7169297
  Vijay Korthikanti authored Jan 22, 2021
  
  a7169297
13 Jan, 2021 1 commit
- Adressing more review comments · 4ae54b55
  Vijay Korthikanti authored Jan 12, 2021
  
  4ae54b55
09 Jan, 2021 1 commit
- decoder support in transformers · 4b506832
  Vijay Korthikanti authored Jan 08, 2021
  
  4b506832
08 Jan, 2021 2 commits
- minor cleanup · a75f1783
  Vijay Korthikanti authored Jan 08, 2021
  
  a75f1783
- vision transformer model and vision classification task · 456f1728
  Vijay Korthikanti authored Jan 08, 2021
  
  456f1728
07 Jan, 2021 1 commit
- Change some arguments to default to on. · 9acc8956
  Jared Casper authored Jan 06, 2021
  
  9acc8956
04 Jan, 2021 1 commit
- Use batched send and recv instead of torch.distributed.ring_exchange() · d899988e
  Deepak Narayanan authored Jan 04, 2021
  
  d899988e
25 Dec, 2020 1 commit
- fp32 is also working · 2eaa3ccc
  mohammad authored Dec 24, 2020
  
  2eaa3ccc
22 Dec, 2020 3 commits
- Address Deepak's comments · 83671bbf
  mshoeybi authored Dec 21, 2020
  
  83671bbf
- Add residual connection in fp32 machinery · 8bed1d63
  mohammad authored Dec 21, 2020
  
  8bed1d63
- Add the option for fp32 residual connection (fp32 residual connection... · 62632d39
  mohammad authored Dec 21, 2020
```
Add the option for fp32 residual connection (fp32 residual connection machinery still needs to be added)
```
  62632d39
19 Dec, 2020 10 commits
- Fix some bugs, add exit-duration capability · a31833ce
  mshoeybi authored Dec 11, 2020
  
  a31833ce
- Nicer error messages for deprecated arguments · 2623551d
  Jared Casper authored Dec 10, 2020
  
  2623551d
- Change lr-warmup-percent to lr-warmup-fraction · 9321d5c6
  Jared Casper authored Dec 10, 2020
  
  9321d5c6
- Work batch-size name changes into task code · 3afcba6e
  Jared Casper authored Dec 09, 2020
  
  3afcba6e
- Address Jared's comments · 00ac56ab
  mohammad authored Dec 09, 2020
  
  00ac56ab
- Sample based learning rate computation · 22ab91bb
  mohammad authored Dec 08, 2020
  
  22ab91bb
- Minor refactoring · c30ba0f7
  mohammad authored Dec 08, 2020
  
  c30ba0f7
- Add constant num micro-batches calculator · feecd5d9
  mohammad authored Dec 07, 2020
  
  feecd5d9
- Add micro-batch size calculator · 6ea23928
  mohammad authored Dec 06, 2020
  
  6ea23928
- Rename --batch-size to --micro-batch-size and drop in-minibatch from... · 9019bbf4
  mohammad authored Dec 06, 2020
```
Rename --batch-size to --micro-batch-size and drop in-minibatch from --num-micro-batches-in-minibatch
```
  9019bbf4
02 Dec, 2020 2 commits
- addressed Jareds comments · fa80af26
  mohammad authored Dec 02, 2020
  
  fa80af26
- addressed Jareds comments · 98989693
  mohammad authored Dec 02, 2020
  
  98989693
29 Nov, 2020 1 commit
- implemented blending datasets · 65290033
  mohammad authored Nov 28, 2020
  
  65290033
28 Nov, 2020 1 commit
- added consumed tokens to checkpoints and some refactoring · f0a445fa
  mohammad authored Nov 27, 2020
  
  f0a445fa
12 Nov, 2020 4 commits
- Throw exception if ring_exchange is not available when pipeline_model_parallel_size > 1 · 2d8de296
  Deepak Narayanan authored Oct 30, 2020
  
  2d8de296
- Remove unused parameter sharing logic · 1271fd73
  Deepak Narayanan authored Oct 28, 2020
  
  1271fd73
- Refactor communication code in main training loop to helper method · 318d68c2
  Deepak Narayanan authored Oct 23, 2020
  
  318d68c2
- Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
  Deepak Narayanan authored Oct 20, 2020
  
  52a5f2f2