Commits · 43d307d5cf2f313e1a53122eb2a448d579111d21 · OpenDAS / Megatron-LM

26 Mar, 2021 1 commit
- added parallelism for computing jaccard similaity · 43d307d5
  mpatwary authored Mar 25, 2021
  
  43d307d5
19 Mar, 2021 1 commit
- Added more feature in train data deduplication · 882683dc
  Mostofa Patwary authored Mar 19, 2021
  
  882683dc
22 Feb, 2021 1 commit
- addressed the comments · ebc4a408
  Mostofa Patwary authored Feb 22, 2021
  
  ebc4a408
19 Feb, 2021 3 commits
- now multiple pickle files can be added · 4b821fb5
  Mostofa Patwary authored Feb 19, 2021
  
  4b821fb5
- Added an example · f6ddeca0
  Mostofa Patwary authored Feb 18, 2021
  
  f6ddeca0
- added storing and loading of pickle file of hash · 2f6e46bd
  Mostofa Patwary authored Feb 18, 2021
  
  2f6e46bd
16 Apr, 2020 1 commit
- changed licence 2019 to 2020 · 463d1257
  Mohammad authored Apr 16, 2020
  
  463d1257
14 Apr, 2020 1 commit
- Move openwebtext scripts to tools directory. · 0dc10d09
  Jared Casper authored Apr 14, 2020
  
  0dc10d09
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e