Commits · 145d2eb947a614fdf81e73efd0cf417c1bc5ac94 · OpenDAS / Megatron-LM

27 Jan, 2022 1 commit

[README] specify explicitly which gpu and node size was used · 145d2eb9

Stas Bekman authored Jan 26, 2022

The paper has this info, so proposing to copy it next to the table. 

Otherwise it's hard to guess whether you used 40GB A100s or 80GB ones (and secondary, n_gpus per node).

Thank you!

145d2eb9

30 Nov, 2021 1 commit
- Mention NeMo Megatron in README · 84529b20
  Jared Casper authored Nov 30, 2021
  
  84529b20
18 Oct, 2021 1 commit
- Clarify README regarding benchmarks. · f482bc50
  Jared Casper authored Oct 18, 2021
  
  f482bc50
15 Oct, 2021 1 commit
- removed greedy argument · 71359e1f
  mshoeybi authored Oct 15, 2021
  
  71359e1f
08 Oct, 2021 1 commit
- Update text generation section of README. Fix line continuation in example scripts. · a2cde4b3
  Jared Casper authored Oct 08, 2021
  
  a2cde4b3
29 Sep, 2021 1 commit
- Remove outdated packaging files. · 6c00f448
  Jared Casper authored Sep 29, 2021
  
  6c00f448
19 Aug, 2021 2 commits
- update readme and arguement definition · 99f47676
  slym authored Aug 19, 2021
  
  99f47676
- Checkpoint a set number of invidividual Transformer layers · c1e0689d
  slym authored Aug 10, 2021
```
consider the case of pipeline-model prallelism

clean up arugments

argument naming cleanup

update readme and examples
```
  c1e0689d
16 Jul, 2021 1 commit
- fix a typo · 61fcc028
  Haibin Lin authored Jul 16, 2021
  
  61fcc028
12 Jul, 2021 2 commits
- Address Mohammad's comment · a037a9c4
  Deepak Narayanan authored Jul 12, 2021
  
  a037a9c4
- Update README to have a small note about interleaved schedule · ab9a7979
  Deepak Narayanan authored Jul 12, 2021
  
  ab9a7979
18 May, 2021 2 commits
- t5 fixes · 2dae74b6
  Stas Bekman authored May 18, 2021
  
  2dae74b6
- Typo fix in readme · 07ca952e
  Devrim authored May 18, 2021
  
  07ca952e
15 May, 2021 1 commit
- One more quick fix. · 657d7cdc
  Jared Casper authored May 14, 2021
  
  657d7cdc
14 May, 2021 3 commits
- Quick fix of copy/paste error. · edb76abe
  Jared Casper authored May 14, 2021
  
  edb76abe
- Adding T5 to docs and a bit of cleanup. · 306eb24e
  Jared Casper authored May 14, 2021
  
  306eb24e
- fixed compatiblity with v0 checkpoints · 26b49aab
  mohammad authored May 14, 2021
  
  26b49aab
12 Apr, 2021 1 commit
- added link to the pipeline papers · 9ec547c9
  mohammad authored Apr 11, 2021
  
  9ec547c9
08 Apr, 2021 1 commit
- Release fixes · 50a4b5fa
  Mohammad Shoeybi authored Apr 08, 2021
  
  50a4b5fa
27 Jan, 2021 1 commit
- Teach merge_mp_partitions how to write out a pipelined model. · 7cabbe67
  Jared Casper authored Jan 27, 2021
  
  7cabbe67
22 Jan, 2021 1 commit
- Fixing merge_mp_partitions · 78066ab0
  Jared Casper authored Jan 20, 2021
  
  78066ab0
12 Jan, 2021 2 commits
- TOC fix · 31733784
  Jared Casper authored Jan 12, 2021
  
  31733784
- Readme update + change gpt2 to gpt · 152aab30
  Mohammad Shoeybi authored Jan 11, 2021
  
  152aab30
19 Dec, 2020 1 commit
- Add pipelining to GLUE and RACE tasks · caa9dca5
  Jared Casper authored Nov 30, 2020
  
  caa9dca5
13 Nov, 2020 1 commit
- New example scripts showing how to use MP, and some notes to main README · a5a41922
  Deepak Narayanan authored Nov 13, 2020
  
  a5a41922
12 Nov, 2020 3 commits
- Address Jared's comments in README and loss_scaler.py · dd079406
  Deepak Narayanan authored Oct 27, 2020
  
  dd079406
- Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
  Deepak Narayanan authored Oct 20, 2020
  
  52a5f2f2
- Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90
  Deepak Narayanan authored Aug 29, 2020
```
Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments
```
  7abd3e90
11 Sep, 2020 1 commit
- Update scaling numbers in README and other small tweaks. · 52819194
  Jared Casper authored Sep 11, 2020
  
  52819194
23 Jul, 2020 1 commit
- Add more documentation · c2a32e12
  Neel Kant authored Jul 22, 2020
  
  c2a32e12
07 Jul, 2020 1 commit
- Addressed Jared's comments · 8d7f508a
  Neel Kant authored Jul 06, 2020
  
  8d7f508a
29 Jun, 2020 1 commit
- Update README · 68283d93
  Neel Kant authored Jun 28, 2020
  
  68283d93
24 Jun, 2020 2 commits
- Update README · 7b3baaaa
  Neel Kant authored Jun 24, 2020
  
  7b3baaaa
- Update README · 47996737
  Neel Kant authored Jun 24, 2020
  
  47996737
15 May, 2020 1 commit
- update · faa462fa
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  faa462fa
14 May, 2020 4 commits
- update · 5f871088
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  5f871088
- update · 1f39e69b
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  1f39e69b
- update2 · 73fef4fc
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  73fef4fc
- six, regex, apex added to requirements · c15829b5
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  c15829b5
05 May, 2020 1 commit
- Fix typos · 4e2b7b26
  David E. Weekly authored May 05, 2020
```
Fixed some small typos in the README
```
  4e2b7b26