Commits · 7a77abd9b6267dc0020a60b424b4748fc22790bb · OpenDAS / Megatron-LM

12 Jan, 2022 1 commit
- Phase1 merge: vit optimizations + dataset enhancements + scaled_softmax kernel · 7a77abd9
  Vijay Korthikanti authored Jan 12, 2022
  
  7a77abd9
17 Aug, 2021 1 commit
- minor changes from github issues · 60078cf3
  mshoeybi authored Aug 17, 2021
  
  60078cf3
19 May, 2021 1 commit
- updated the evaluation script for retriver · f21a6628
  Mostofa Patwary authored May 18, 2021
  
  f21a6628
18 May, 2021 1 commit
- before cleaning the comments · 7a0710ec
  Mostofa Patwary authored May 18, 2021
  
  7a0710ec
14 May, 2021 1 commit
- Ensure numpy random seed is within range. · 1c4c360f
  Jared Casper authored May 14, 2021
  
  1c4c360f
16 Apr, 2021 1 commit
- Integrate code from t5_main into existing code. · 48a5e0da
  Jared Casper authored Apr 15, 2021
  
  48a5e0da
09 Apr, 2021 1 commit
- implementing DPR · 10ff0607
  Mostofa Patwary authored Apr 09, 2021
  
  10ff0607
19 Mar, 2021 1 commit
- ICT zeroshot evaluation · fcfd0949
  Mostofa Patwary authored Mar 19, 2021
  
  fcfd0949
10 Mar, 2021 1 commit
- ICT zeroshot evaluation code · 8b04e0e4
  Mostofa Patwary authored Mar 10, 2021
  
  8b04e0e4
23 Feb, 2021 1 commit
- Added code for building embeddings and savings · bcd605f8
  Mostofa Patwary authored Feb 23, 2021
  
  bcd605f8
18 Feb, 2021 1 commit
- evaluation code ongoing · 612f438a
  Mostofa Patwary authored Feb 17, 2021
  
  612f438a
10 Feb, 2021 1 commit
- removed the function compile_helper · 22a3d81a
  Mostofa Patwary authored Feb 09, 2021
  
  22a3d81a
04 Feb, 2021 1 commit
- remove the function get_one_epoch_dataloader and also added assert · 8eff2a99
  Mostofa Patwary authored Feb 03, 2021
  
  8eff2a99
29 Jan, 2021 2 commits
- WIP: main_retriver_merge · 0295bb89
  Mostofa Patwary authored Jan 29, 2021
  
  0295bb89
- WIP: main_retriver_merge · 17d897e0
  Mostofa Patwary authored Jan 29, 2021
  
  17d897e0
28 Jan, 2021 1 commit
- license text for autoaugmentation · 54bb3046
  Vijay Korthikanti authored Jan 28, 2021
  
  54bb3046
26 Jan, 2021 1 commit
- dataloader_type argument fix + randomsampler fix · 85589322
  Vijay Korthikanti authored Jan 25, 2021
  
  85589322
25 Jan, 2021 1 commit
- Adding option to remove the binary head for BERT · dcff1acd
  Mohammad Shoeybi authored Jan 25, 2021
  
  dcff1acd
22 Jan, 2021 1 commit
- Addressing review comments · a7169297
  Vijay Korthikanti authored Jan 22, 2021
  
  a7169297
12 Jan, 2021 1 commit
- Readme update + change gpt2 to gpt · 152aab30
  Mohammad Shoeybi authored Jan 11, 2021
  
  152aab30
08 Jan, 2021 1 commit
- vision transformer model and vision classification task · 456f1728
  Vijay Korthikanti authored Jan 08, 2021
  
  456f1728
30 Dec, 2020 1 commit
- moved compile helper to initialize · 242770dd
  mshoeybi authored Dec 29, 2020
  
  242770dd
19 Dec, 2020 7 commits
- Added a comment to justify 80 percent · 25c07e14
  mohammad authored Dec 14, 2020
  
  25c07e14
- Some bugfixes · 29a69547
  mshoeybi authored Dec 11, 2020
  
  29a69547
- Last epoch should not be globally shuffled · 39181113
  mshoeybi authored Dec 11, 2020
  
  39181113
- Fix some bugs, add exit-duration capability · a31833ce
  mshoeybi authored Dec 11, 2020
  
  a31833ce
- Add constant num micro-batches calculator · feecd5d9
  mohammad authored Dec 07, 2020
  
  feecd5d9
- Add micro-batch size calculator · 6ea23928
  mohammad authored Dec 06, 2020
  
  6ea23928
- Rename --batch-size to --micro-batch-size and drop in-minibatch from... · 9019bbf4
  mohammad authored Dec 06, 2020
```
Rename --batch-size to --micro-batch-size and drop in-minibatch from --num-micro-batches-in-minibatch
```
  9019bbf4
02 Dec, 2020 1 commit
- addressed Jareds comments · 98989693
  mohammad authored Dec 02, 2020
  
  98989693
29 Nov, 2020 2 commits
- implemented blending datasets · 65290033
  mohammad authored Nov 28, 2020
  
  65290033
- added blendable dataset · d3bb1a06
  mohammad authored Nov 28, 2020
  
  d3bb1a06
28 Nov, 2020 1 commit
- added consumed tokens to checkpoints and some refactoring · f0a445fa
  mohammad authored Nov 27, 2020
  
  f0a445fa
26 Nov, 2020 1 commit
- simplified sampler · 4311b695
  mohammad authored Nov 25, 2020
  
  4311b695
12 Nov, 2020 2 commits

Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
Deepak Narayanan authored Oct 20, 2020

52a5f2f2

Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90

Deepak Narayanan authored Aug 29, 2020

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

7abd3e90

26 Jul, 2020 1 commit
- Add additional assertion on Indexer to test correctness, and limit verbosity in other classes · eaa5d877
  Neel Kant authored Jul 25, 2020
  
  eaa5d877
23 Jul, 2020 1 commit
- Address comments from last week · 11f76cd3
  Neel Kant authored Jul 22, 2020
  
  11f76cd3
14 Jul, 2020 2 commits
- Fix gpt2_dataset import error · 5247f24c
  Neel Kant authored Jul 14, 2020
  
  5247f24c
- Additional refactoring · b9bd1a11
  Neel Kant authored Jul 13, 2020
  
  b9bd1a11