Commits · 651cc107d7e4070e8661ccaf19cd9f4f283e0dd0 · OpenDAS / Megatron-LM

29 Jan, 2021 2 commits
- WIP: main_retriver_merge · 0295bb89
  Mostofa Patwary authored Jan 29, 2021
  
  0295bb89
- WIP: main_retriver_merge · 17d897e0
  Mostofa Patwary authored Jan 29, 2021
  
  17d897e0
28 Jan, 2021 1 commit
- license text for autoaugmentation · 54bb3046
  Vijay Korthikanti authored Jan 28, 2021
  
  54bb3046
26 Jan, 2021 1 commit
- dataloader_type argument fix + randomsampler fix · 85589322
  Vijay Korthikanti authored Jan 25, 2021
  
  85589322
25 Jan, 2021 1 commit
- Adding option to remove the binary head for BERT · dcff1acd
  Mohammad Shoeybi authored Jan 25, 2021
  
  dcff1acd
22 Jan, 2021 1 commit
- Addressing review comments · a7169297
  Vijay Korthikanti authored Jan 22, 2021
  
  a7169297
12 Jan, 2021 1 commit
- Readme update + change gpt2 to gpt · 152aab30
  Mohammad Shoeybi authored Jan 11, 2021
  
  152aab30
08 Jan, 2021 1 commit
- vision transformer model and vision classification task · 456f1728
  Vijay Korthikanti authored Jan 08, 2021
  
  456f1728
30 Dec, 2020 1 commit
- moved compile helper to initialize · 242770dd
  mshoeybi authored Dec 29, 2020
  
  242770dd
19 Dec, 2020 7 commits
- Added a comment to justify 80 percent · 25c07e14
  mohammad authored Dec 14, 2020
  
  25c07e14
- Some bugfixes · 29a69547
  mshoeybi authored Dec 11, 2020
  
  29a69547
- Last epoch should not be globally shuffled · 39181113
  mshoeybi authored Dec 11, 2020
  
  39181113
- Fix some bugs, add exit-duration capability · a31833ce
  mshoeybi authored Dec 11, 2020
  
  a31833ce
- Add constant num micro-batches calculator · feecd5d9
  mohammad authored Dec 07, 2020
  
  feecd5d9
- Add micro-batch size calculator · 6ea23928
  mohammad authored Dec 06, 2020
  
  6ea23928
- Rename --batch-size to --micro-batch-size and drop in-minibatch from... · 9019bbf4
  mohammad authored Dec 06, 2020
```
Rename --batch-size to --micro-batch-size and drop in-minibatch from --num-micro-batches-in-minibatch
```
  9019bbf4
02 Dec, 2020 1 commit
- addressed Jareds comments · 98989693
  mohammad authored Dec 02, 2020
  
  98989693
29 Nov, 2020 2 commits
- implemented blending datasets · 65290033
  mohammad authored Nov 28, 2020
  
  65290033
- added blendable dataset · d3bb1a06
  mohammad authored Nov 28, 2020
  
  d3bb1a06
28 Nov, 2020 1 commit
- added consumed tokens to checkpoints and some refactoring · f0a445fa
  mohammad authored Nov 27, 2020
  
  f0a445fa
26 Nov, 2020 1 commit
- simplified sampler · 4311b695
  mohammad authored Nov 25, 2020
  
  4311b695
12 Nov, 2020 2 commits

Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
Deepak Narayanan authored Oct 20, 2020

52a5f2f2

Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90

Deepak Narayanan authored Aug 29, 2020

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

7abd3e90

26 Jul, 2020 1 commit
- Add additional assertion on Indexer to test correctness, and limit verbosity in other classes · eaa5d877
  Neel Kant authored Jul 25, 2020
  
  eaa5d877
23 Jul, 2020 1 commit
- Address comments from last week · 11f76cd3
  Neel Kant authored Jul 22, 2020
  
  11f76cd3
14 Jul, 2020 2 commits
- Fix gpt2_dataset import error · 5247f24c
  Neel Kant authored Jul 14, 2020
  
  5247f24c
- Additional refactoring · b9bd1a11
  Neel Kant authored Jul 13, 2020
  
  b9bd1a11
07 Jul, 2020 1 commit
- Addressed Jared's comments · 8d7f508a
  Neel Kant authored Jul 06, 2020
  
  8d7f508a
02 Jul, 2020 1 commit
- One sentence options · 03feecbc
  Neel Kant authored Jul 02, 2020
  
  03feecbc
30 Jun, 2020 1 commit
- adding realm_index.py · d9d4ce70
  Neel Kant authored Jun 30, 2020
  
  d9d4ce70
29 Jun, 2020 1 commit
- Changes after running on draco · 44860f8d
  Neel Kant authored Jun 28, 2020
  
  44860f8d
28 Jun, 2020 2 commits
- remove data/preprocess_data.py · 8f3f338a
  Neel Kant authored Jun 27, 2020
  
  8f3f338a
- remove ict_dataset.py · a14f1442
  Neel Kant authored Jun 27, 2020
  
  a14f1442
26 Jun, 2020 1 commit
- Cosmetic changes · 2a3b445d
  Neel Kant authored Jun 26, 2020
  
  2a3b445d
25 Jun, 2020 1 commit
- Add stanza NER salient span masking · e7045139
  Neel Kant authored Jun 24, 2020
  
  e7045139
24 Jun, 2020 2 commits
- Fix accidental comment · 1f514406
  Neel Kant authored Jun 23, 2020
  
  1f514406
- More ict_merge changes and interactive testing · 3354081f
  Neel Kant authored Jun 23, 2020
  
  3354081f
22 Jun, 2020 1 commit
- Address most of comments from code review w/ Mohammad · 6495927e
  Neel Kant authored Jun 22, 2020
  
  6495927e
15 Jun, 2020 1 commit
- Debugging done on Circe · b8bb0b49
  Mohammad authored Jun 15, 2020
  
  b8bb0b49
10 Jun, 2020 1 commit
- gpt2 mmap sample map · dd76c914
  Raul Puri authored Jun 10, 2020
  
  dd76c914