Commits · 0747e8e558525f4db34d16163b873142d0c61790 · OpenDAS / Megatron-LM

27 Jan, 2022 2 commits
- add clarification about the model parallel size column · 2d767f48
  Stas Bekman authored Jan 26, 2022
  
  2d767f48
- [README] specify explicitly which gpu and node size was used · 145d2eb9
  Stas Bekman authored Jan 26, 2022
```
The paper has this info, so proposing to copy it next to the table. 

Otherwise it's hard to guess whether you used 40GB A100s or 80GB ones (and secondary, n_gpus per node).

Thank you!
```
  145d2eb9
13 Dec, 2021 1 commit
- Update README.md · 5352377c
  Rajesh Koilpillai authored Dec 13, 2021
  
  5352377c
30 Nov, 2021 1 commit
- Mention NeMo Megatron in README · 84529b20
  Jared Casper authored Nov 30, 2021
  
  84529b20
18 Oct, 2021 1 commit
- Clarify README regarding benchmarks. · f482bc50
  Jared Casper authored Oct 18, 2021
  
  f482bc50
15 Oct, 2021 1 commit
- removed greedy argument · 71359e1f
  mshoeybi authored Oct 15, 2021
  
  71359e1f
08 Oct, 2021 1 commit
- Update text generation section of README. Fix line continuation in example scripts. · a2cde4b3
  Jared Casper authored Oct 08, 2021
  
  a2cde4b3
29 Sep, 2021 1 commit
- Remove outdated packaging files. · 6c00f448
  Jared Casper authored Sep 29, 2021
  
  6c00f448
19 Aug, 2021 2 commits
- update readme and arguement definition · 99f47676
  slym authored Aug 19, 2021
  
  99f47676
- Checkpoint a set number of invidividual Transformer layers · c1e0689d
  slym authored Aug 10, 2021
```
consider the case of pipeline-model prallelism

clean up arugments

argument naming cleanup

update readme and examples
```
  c1e0689d
16 Jul, 2021 1 commit
- fix a typo · 61fcc028
  Haibin Lin authored Jul 16, 2021
  
  61fcc028
12 Jul, 2021 2 commits
- Address Mohammad's comment · a037a9c4
  Deepak Narayanan authored Jul 12, 2021
  
  a037a9c4
- Update README to have a small note about interleaved schedule · ab9a7979
  Deepak Narayanan authored Jul 12, 2021
  
  ab9a7979
18 May, 2021 2 commits
- t5 fixes · 2dae74b6
  Stas Bekman authored May 18, 2021
  
  2dae74b6
- Typo fix in readme · 07ca952e
  Devrim authored May 18, 2021
  
  07ca952e
15 May, 2021 1 commit
- One more quick fix. · 657d7cdc
  Jared Casper authored May 14, 2021
  
  657d7cdc
14 May, 2021 3 commits
- Quick fix of copy/paste error. · edb76abe
  Jared Casper authored May 14, 2021
  
  edb76abe
- Adding T5 to docs and a bit of cleanup. · 306eb24e
  Jared Casper authored May 14, 2021
  
  306eb24e
- fixed compatiblity with v0 checkpoints · 26b49aab
  mohammad authored May 14, 2021
  
  26b49aab
12 Apr, 2021 1 commit
- added link to the pipeline papers · 9ec547c9
  mohammad authored Apr 11, 2021
  
  9ec547c9
08 Apr, 2021 1 commit
- Release fixes · 50a4b5fa
  Mohammad Shoeybi authored Apr 08, 2021
  
  50a4b5fa
27 Jan, 2021 1 commit
- Teach merge_mp_partitions how to write out a pipelined model. · 7cabbe67
  Jared Casper authored Jan 27, 2021
  
  7cabbe67
22 Jan, 2021 1 commit
- Fixing merge_mp_partitions · 78066ab0
  Jared Casper authored Jan 20, 2021
  
  78066ab0
12 Jan, 2021 2 commits
- TOC fix · 31733784
  Jared Casper authored Jan 12, 2021
  
  31733784
- Readme update + change gpt2 to gpt · 152aab30
  Mohammad Shoeybi authored Jan 11, 2021
  
  152aab30
19 Dec, 2020 1 commit
- Add pipelining to GLUE and RACE tasks · caa9dca5
  Jared Casper authored Nov 30, 2020
  
  caa9dca5
13 Nov, 2020 1 commit
- New example scripts showing how to use MP, and some notes to main README · a5a41922
  Deepak Narayanan authored Nov 13, 2020
  
  a5a41922
12 Nov, 2020 3 commits
- Address Jared's comments in README and loss_scaler.py · dd079406
  Deepak Narayanan authored Oct 27, 2020
  
  dd079406
- Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
  Deepak Narayanan authored Oct 20, 2020
  
  52a5f2f2
- Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90
  Deepak Narayanan authored Aug 29, 2020
```
Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments
```
  7abd3e90
11 Sep, 2020 1 commit
- Update scaling numbers in README and other small tweaks. · 52819194
  Jared Casper authored Sep 11, 2020
  
  52819194
23 Jul, 2020 1 commit
- Add more documentation · c2a32e12
  Neel Kant authored Jul 22, 2020
  
  c2a32e12
07 Jul, 2020 1 commit
- Addressed Jared's comments · 8d7f508a
  Neel Kant authored Jul 06, 2020
  
  8d7f508a
29 Jun, 2020 1 commit
- Update README · 68283d93
  Neel Kant authored Jun 28, 2020
  
  68283d93
24 Jun, 2020 2 commits
- Update README · 7b3baaaa
  Neel Kant authored Jun 24, 2020
  
  7b3baaaa
- Update README · 47996737
  Neel Kant authored Jun 24, 2020
  
  47996737
15 May, 2020 1 commit
- update · faa462fa
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  faa462fa
14 May, 2020 3 commits
- update · 5f871088
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  5f871088
- update · 1f39e69b
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  1f39e69b
- update2 · 73fef4fc
  Evelina Bakhturina authored May 14, 2020
```
Signed-off-by: Evelina Bakhturina <ebakhturina@nvidia.com>
```
  73fef4fc