Commits · a2c5e6cd58728190e1183fd02fdabbbc57e35f0f · OpenDAS / Megatron-LM

12 May, 2022 1 commit

Data preprocessing testing changes + fixes · a2c5e6cd

John Kamalu authored May 12, 2022

tools/merge_datasets.py
- tool to merge multiple dataset files into a single dataset
- testing conducted and included in the megatron-testing repo https://gitlab-master.nvidia.com/ADLR/megatron-testing

tools/preprocess_data.py
- magic numbers changed to required command line arguments

megatron/data/indexed_dataset.py
- when merging, fix to properly update document index
- testing conducted and included in the megatron-testing repo (see above)
- fix follows this history https://github.com/bigscience-workshop/Megatron-DeepSpeed/pull/66

a2c5e6cd

15 Feb, 2022 3 commits
- more minor fixes · 1cd3650d
  Vijay Korthikanti authored Feb 01, 2022
  
  1cd3650d
- minor fixes · 6f3bf9c0
  Vijay Korthikanti authored Feb 01, 2022
  
  6f3bf9c0
- vision third phase merge: pretraining methods + mit,swin backbones · 48c2a144
  Vijay Korthikanti authored Jan 31, 2022
  
  48c2a144
01 Feb, 2022 3 commits
- more minor fixes · 3f1a728a
  Vijay Korthikanti authored Feb 01, 2022
  
  3f1a728a
- minor fixes · e1f9c3a5
  Vijay Korthikanti authored Feb 01, 2022
  
  e1f9c3a5
- vision third phase merge: pretraining methods + mit,swin backbones · 01a82723
  Vijay Korthikanti authored Jan 31, 2022
  
  01a82723
14 Jan, 2022 1 commit
- address review comments · 343dc97a
  Vijay Korthikanti authored Jan 14, 2022
  
  343dc97a
13 Jan, 2022 1 commit
- minor fixes · 3b29f0e7
  Vijay Korthikanti authored Jan 13, 2022
  
  3b29f0e7
12 Jan, 2022 1 commit
- Phase1 merge: vit optimizations + dataset enhancements + scaled_softmax kernel · 7a77abd9
  Vijay Korthikanti authored Jan 12, 2022
  
  7a77abd9
17 Aug, 2021 1 commit
- minor changes from github issues · 60078cf3
  mshoeybi authored Aug 17, 2021
  
  60078cf3
19 May, 2021 1 commit
- updated the evaluation script for retriver · f21a6628
  Mostofa Patwary authored May 18, 2021
  
  f21a6628
18 May, 2021 1 commit
- before cleaning the comments · 7a0710ec
  Mostofa Patwary authored May 18, 2021
  
  7a0710ec
14 May, 2021 1 commit
- Ensure numpy random seed is within range. · 1c4c360f
  Jared Casper authored May 14, 2021
  
  1c4c360f
16 Apr, 2021 1 commit
- Integrate code from t5_main into existing code. · 48a5e0da
  Jared Casper authored Apr 15, 2021
  
  48a5e0da
09 Apr, 2021 1 commit
- implementing DPR · 10ff0607
  Mostofa Patwary authored Apr 09, 2021
  
  10ff0607
19 Mar, 2021 1 commit
- ICT zeroshot evaluation · fcfd0949
  Mostofa Patwary authored Mar 19, 2021
  
  fcfd0949
10 Mar, 2021 1 commit
- ICT zeroshot evaluation code · 8b04e0e4
  Mostofa Patwary authored Mar 10, 2021
  
  8b04e0e4
23 Feb, 2021 1 commit
- Added code for building embeddings and savings · bcd605f8
  Mostofa Patwary authored Feb 23, 2021
  
  bcd605f8
18 Feb, 2021 1 commit
- evaluation code ongoing · 612f438a
  Mostofa Patwary authored Feb 17, 2021
  
  612f438a
10 Feb, 2021 1 commit
- removed the function compile_helper · 22a3d81a
  Mostofa Patwary authored Feb 09, 2021
  
  22a3d81a
04 Feb, 2021 1 commit
- remove the function get_one_epoch_dataloader and also added assert · 8eff2a99
  Mostofa Patwary authored Feb 03, 2021
  
  8eff2a99
29 Jan, 2021 2 commits
- WIP: main_retriver_merge · 0295bb89
  Mostofa Patwary authored Jan 29, 2021
  
  0295bb89
- WIP: main_retriver_merge · 17d897e0
  Mostofa Patwary authored Jan 29, 2021
  
  17d897e0
28 Jan, 2021 1 commit
- license text for autoaugmentation · 54bb3046
  Vijay Korthikanti authored Jan 28, 2021
  
  54bb3046
26 Jan, 2021 1 commit
- dataloader_type argument fix + randomsampler fix · 85589322
  Vijay Korthikanti authored Jan 25, 2021
  
  85589322
25 Jan, 2021 1 commit
- Adding option to remove the binary head for BERT · dcff1acd
  Mohammad Shoeybi authored Jan 25, 2021
  
  dcff1acd
22 Jan, 2021 1 commit
- Addressing review comments · a7169297
  Vijay Korthikanti authored Jan 22, 2021
  
  a7169297
12 Jan, 2021 1 commit
- Readme update + change gpt2 to gpt · 152aab30
  Mohammad Shoeybi authored Jan 11, 2021
  
  152aab30
08 Jan, 2021 1 commit
- vision transformer model and vision classification task · 456f1728
  Vijay Korthikanti authored Jan 08, 2021
  
  456f1728
30 Dec, 2020 1 commit
- moved compile helper to initialize · 242770dd
  mshoeybi authored Dec 29, 2020
  
  242770dd
19 Dec, 2020 7 commits
- Added a comment to justify 80 percent · 25c07e14
  mohammad authored Dec 14, 2020
  
  25c07e14
- Some bugfixes · 29a69547
  mshoeybi authored Dec 11, 2020
  
  29a69547
- Last epoch should not be globally shuffled · 39181113
  mshoeybi authored Dec 11, 2020
  
  39181113
- Fix some bugs, add exit-duration capability · a31833ce
  mshoeybi authored Dec 11, 2020
  
  a31833ce
- Add constant num micro-batches calculator · feecd5d9
  mohammad authored Dec 07, 2020
  
  feecd5d9
- Add micro-batch size calculator · 6ea23928
  mohammad authored Dec 06, 2020
  
  6ea23928
- Rename --batch-size to --micro-batch-size and drop in-minibatch from... · 9019bbf4
  mohammad authored Dec 06, 2020
```
Rename --batch-size to --micro-batch-size and drop in-minibatch from --num-micro-batches-in-minibatch
```
  9019bbf4
02 Dec, 2020 1 commit
- addressed Jareds comments · 98989693
  mohammad authored Dec 02, 2020
  
  98989693
29 Nov, 2020 1 commit
- implemented blending datasets · 65290033
  mohammad authored Nov 28, 2020
  
  65290033