Commits · 21a11ca02bb37dc173328883898d34aba043f74e · OpenDAS / Megatron-LM

12 Jul, 2021 2 commits
- Address Mohammad's comment · a037a9c4
  Deepak Narayanan authored Jul 12, 2021
  
  a037a9c4
- Update README to have a small note about interleaved schedule · ab9a7979
  Deepak Narayanan authored Jul 12, 2021
  
  ab9a7979
18 May, 2021 2 commits
- t5 fixes · 2dae74b6
  Stas Bekman authored May 18, 2021
  
  2dae74b6
- Typo fix in readme · 07ca952e
  Devrim authored May 18, 2021
  
  07ca952e
15 May, 2021 1 commit
- One more quick fix. · 657d7cdc
  Jared Casper authored May 14, 2021
  
  657d7cdc
14 May, 2021 3 commits
- Quick fix of copy/paste error. · edb76abe
  Jared Casper authored May 14, 2021
  
  edb76abe
- Adding T5 to docs and a bit of cleanup. · 306eb24e
  Jared Casper authored May 14, 2021
  
  306eb24e
- fixed compatiblity with v0 checkpoints · 26b49aab
  mohammad authored May 14, 2021
  
  26b49aab
12 Apr, 2021 1 commit
- added link to the pipeline papers · 9ec547c9
  mohammad authored Apr 11, 2021
  
  9ec547c9
08 Apr, 2021 1 commit
- Release fixes · 50a4b5fa
  Mohammad Shoeybi authored Apr 08, 2021
  
  50a4b5fa
27 Jan, 2021 1 commit
- Teach merge_mp_partitions how to write out a pipelined model. · 7cabbe67
  Jared Casper authored Jan 27, 2021
  
  7cabbe67
22 Jan, 2021 1 commit
- Fixing merge_mp_partitions · 78066ab0
  Jared Casper authored Jan 20, 2021
  
  78066ab0
12 Jan, 2021 2 commits
- TOC fix · 31733784
  Jared Casper authored Jan 12, 2021
  
  31733784
- Readme update + change gpt2 to gpt · 152aab30
  Mohammad Shoeybi authored Jan 11, 2021
  
  152aab30
19 Dec, 2020 1 commit
- Add pipelining to GLUE and RACE tasks · caa9dca5
  Jared Casper authored Nov 30, 2020
  
  caa9dca5
13 Nov, 2020 1 commit
- New example scripts showing how to use MP, and some notes to main README · a5a41922
  Deepak Narayanan authored Nov 13, 2020
  
  a5a41922
12 Nov, 2020 3 commits
- Address Jared's comments in README and loss_scaler.py · dd079406
  Deepak Narayanan authored Oct 27, 2020
  
  dd079406
- Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
  Deepak Narayanan authored Oct 20, 2020
  
  52a5f2f2
- Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90
  Deepak Narayanan authored Aug 29, 2020
```
Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments
```
  7abd3e90
11 Sep, 2020 1 commit
- Update scaling numbers in README and other small tweaks. · 52819194
  Jared Casper authored Sep 11, 2020
  
  52819194
23 Jul, 2020 1 commit
- Add more documentation · c2a32e12
  Neel Kant authored Jul 22, 2020
  
  c2a32e12
07 Jul, 2020 1 commit
- Addressed Jared's comments · 8d7f508a
  Neel Kant authored Jul 06, 2020
  
  8d7f508a
29 Jun, 2020 1 commit
- Update README · 68283d93
  Neel Kant authored Jun 28, 2020
  
  68283d93
24 Jun, 2020 2 commits
- Update README · 7b3baaaa
  Neel Kant authored Jun 24, 2020
  
  7b3baaaa
- Update README · 47996737
  Neel Kant authored Jun 24, 2020
  
  47996737
15 May, 2020 1 commit
- update · faa462fa
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  faa462fa
14 May, 2020 4 commits
- update · 5f871088
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  5f871088
- update · 1f39e69b
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  1f39e69b
- update2 · 73fef4fc
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  73fef4fc
- six, regex, apex added to requirements · c15829b5
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  c15829b5
05 May, 2020 1 commit
- Fix typos · 4e2b7b26
  David E. Weekly authored May 05, 2020
```
Fixed some small typos in the README
```
  4e2b7b26
21 Apr, 2020 1 commit
- Update README.md · 89462697
  Raul Puri authored Apr 20, 2020
  
  89462697
20 Apr, 2020 3 commits
- Move links to download checkpoints section · b46e28bd
  Steven Steinke authored Apr 20, 2020
  
  b46e28bd
- Links to vocab in Data Preprocessing; added "WordPiece" to description · 8e4cb2a6
  Steven Steinke authored Apr 20, 2020
  
  8e4cb2a6
- Links to vocabs and merge table in Data Preprocessing · bdc3d5ba
  Steven Steinke authored Apr 20, 2020
  
  bdc3d5ba
16 Apr, 2020 5 commits
- remove wikitext comment · 7ec42d95
  Steven Steinke authored Apr 16, 2020
  
  7ec42d95
- RACE explained · 4f770782
  Steven Steinke authored Apr 16, 2020
  
  4f770782
- Update README.md · d976f011
  Steven Steinke authored Apr 16, 2020
  
  d976f011
- Fixed minor inconsistencies in scripts, added distributed comment · 2e0b3fca
  Steven Steinke authored Apr 16, 2020
  
  2e0b3fca
- looks great · 3df23c72
  Mohammad authored Apr 16, 2020
  
  3df23c72