Commits · 2be1e51010660c5823c16c81d0bc47eeba219eab · OpenDAS / Megatron-LM

10 Jun, 2021 1 commit
- Adding readme · c45109ed
  Mostofa Patwary authored Jun 09, 2021
  
  c45109ed
07 Jun, 2021 1 commit
- Update T5 scripts · 3dadd16d
  Deepak Narayanan authored Jun 07, 2021
  
  3dadd16d
03 Jun, 2021 1 commit
- updating script · c7c65bbb
  Mostofa Patwary authored Jun 03, 2021
  
  c7c65bbb
20 May, 2021 1 commit
- updating the scripts · 63121a9e
  Mostofa Patwary authored May 19, 2021
  
  63121a9e
19 May, 2021 2 commits
- updated the evaluation script for retriver · dfb6a9b3
  Mostofa Patwary authored May 18, 2021
  
  dfb6a9b3
- Fixed issues with ICT pretraining · 7577931b
  Mostofa Patwary authored May 18, 2021
  
  7577931b
14 May, 2021 1 commit
- Adding T5 to docs and a bit of cleanup. · 306eb24e
  Jared Casper authored May 14, 2021
  
  306eb24e
19 Mar, 2021 1 commit
- ICT zeroshot evaluation · fcfd0949
  Mostofa Patwary authored Mar 19, 2021
  
  fcfd0949
11 Mar, 2021 1 commit
- made more generic, aligned with other tasks · 661553f6
  Mostofa Patwary authored Mar 11, 2021
  
  661553f6
10 Mar, 2021 1 commit
- ICT zeroshot evaluation code · 8b04e0e4
  Mostofa Patwary authored Mar 10, 2021
  
  8b04e0e4
23 Feb, 2021 1 commit
- added script for creating embeddings · edf475ff
  Mostofa Patwary authored Feb 23, 2021
  
  edf475ff
15 Feb, 2021 1 commit
- updated script as well · a4b628ab
  Mostofa Patwary authored Feb 15, 2021
  
  a4b628ab
04 Feb, 2021 1 commit
- updaded examples/pretrain_ict.sh · 27f0f0ef
  Mostofa Patwary authored Feb 03, 2021
  
  27f0f0ef
03 Feb, 2021 1 commit
- added examples/pretrain_ict.sh · 38898931
  Mostofa Patwary authored Feb 03, 2021
  
  38898931
27 Jan, 2021 1 commit
- added init method std · a6bf1a04
  mshoeybi authored Jan 26, 2021
  
  a6bf1a04
12 Jan, 2021 1 commit
- Readme update + change gpt2 to gpt · 152aab30
  Mohammad Shoeybi authored Jan 11, 2021
  
  152aab30
13 Nov, 2020 1 commit
- New example scripts showing how to use MP, and some notes to main README · a5a41922
  Deepak Narayanan authored Nov 13, 2020
  
  a5a41922
12 Nov, 2020 2 commits

Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
Deepak Narayanan authored Oct 20, 2020

52a5f2f2

Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90

Deepak Narayanan authored Aug 29, 2020

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

7abd3e90

28 Apr, 2020 1 commit
- updated the script · 25234f8a
  Mostofa Patwary authored Apr 27, 2020
  
  25234f8a
16 Apr, 2020 6 commits
- Fix embeddings and seq length · 475a6b99
  Steven Steinke authored Apr 16, 2020
  
  475a6b99
- Fix finetine -> tune · 5cb2d1ea
  Steven Steinke authored Apr 16, 2020
  
  5cb2d1ea
- fix typos · 50391a0b
  Steven Steinke authored Apr 16, 2020
  
  50391a0b
- latest · 1f45adcc
  Mohammad authored Apr 16, 2020
  
  1f45adcc
- updated scripts · ca19a8b5
  Mohammad authored Apr 16, 2020
  
  ca19a8b5
- moved steves branch · 78cb1781
  Mohammad authored Apr 16, 2020
  
  78cb1781
15 Apr, 2020 1 commit
- minor cleanup · c0a59a66
  Mohammad authored Apr 14, 2020
  
  c0a59a66
08 Apr, 2020 2 commits
- Move old scripts to deprecated_data_utils. · 836c6776
  Jared Casper authored Apr 08, 2020
  
  836c6776
- Rename and organize scripts directory. · 985f027d
  Jared Casper authored Apr 08, 2020
  
  985f027d