Commits · 1f3f834c7790a6faac5a3887f271d3f020bdf2f4 · OpenDAS / Megatron-LM

29 Sep, 2020 1 commit
- Text generation bugs · 1f3f834c
  Mostofa Patwary authored Sep 28, 2020
  
  1f3f834c
25 Sep, 2020 3 commits
- Merge branch 'log_report_fix' into 'main' · 4bf923d5
  Jared Casper authored Sep 25, 2020
```
fixed logging bug

See merge request ADLR/megatron-lm!134
```
  4bf923d5
- fixed the interation count · 2db01991
  mohammad authored Sep 24, 2020
  
  2db01991
- Merge branch 'main' into log_report_fix · 9787e6c9
  mohammad authored Sep 24, 2020
  
  9787e6c9
24 Sep, 2020 2 commits
- Merge branch 'vijay/softmax_fusion' into 'main' · 2bd27e20
  Jared Casper authored Sep 24, 2020
```
scaling-generalized masking-softmax fusion kernel

See merge request ADLR/megatron-lm!131
```
  2bd27e20
- nan values report added · abd7636e
  mohammad authored Sep 23, 2020
  
  abd7636e
22 Sep, 2020 1 commit
- fixed logging bug · a74f8d9c
  mohammad authored Sep 22, 2020
  
  a74f8d9c
18 Sep, 2020 1 commit
- scaling-generalized masking-softmax fusion kernel · d73def35
  root authored Sep 18, 2020
  
  d73def35
11 Sep, 2020 4 commits
- Merge branch 'readme_update' into 'main' · 19301985
  Jared Casper authored Sep 11, 2020
```
Update scaling numbers in README and other small tweaks.

See merge request ADLR/megatron-lm!130
```
  19301985
- Update scaling numbers in README and other small tweaks. · 52819194
  Jared Casper authored Sep 11, 2020
  
  52819194
- Merge branch 'vijay/softmax_fusion' into 'main' · 91ee60df
  Jared Casper authored Sep 11, 2020
```
Various speed optimizations.

See merge request ADLR/megatron-lm!124
```
  91ee60df
- Various speed optimizations. · 51a2e6b0
  Vijay Korthikanti authored Sep 11, 2020
  
  51a2e6b0
03 Sep, 2020 3 commits
- Merge branch 'dist_act_chkpt' into 'main' · 12518332
  Jared Casper authored Sep 03, 2020
```
added splitting checkpointed activations across model parallel partitions

See merge request ADLR/megatron-lm!121
```
  12518332
- Merge branch 'fix_cpu_init' into 'main' · 930ec4a2
  Jared Casper authored Sep 03, 2020
```
Fix cpu initialization for mp=1 and fp16

See merge request ADLR/megatron-lm!123
```
  930ec4a2
- debuged the cpu init for mp=1 · 04cd82bd
  mohammad authored Sep 03, 2020
  
  04cd82bd
02 Sep, 2020 2 commits
- addressed Jareds comments · 5d29769c
  mohammad authored Sep 02, 2020
  
  5d29769c
- added splitting checkpointed activations · d6c4248b
  mohammad authored Sep 02, 2020
  
  d6c4248b
01 Sep, 2020 1 commit

Merge branch 'onnx-safe' into 'main' · 42d2be06

Mohammad Shoeybi authored Sep 01, 2020

--onnx-safe made overrideable, -lazy_mpu_init act as _USE_CPU_INITIALIZATION

See merge request ADLR/megatron-lm!117

42d2be06

28 Aug, 2020 1 commit
- Added documentation · b6b48a32
  Boris Fomitchev authored Aug 27, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  b6b48a32
27 Aug, 2020 1 commit
- Addressing code review comments · 1eafa861
  Boris Fomitchev authored Aug 26, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  1eafa861
26 Aug, 2020 1 commit
- Making --lazy_mpu_init act as _USE_CPU_INITIALIZATION · e10760ea
  Boris Fomitchev authored Aug 26, 2020
  
  e10760ea
25 Aug, 2020 1 commit
- Making onnx-safe option overrideable · d4bb6cb0
  Boris Fomitchev authored Aug 25, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  d4bb6cb0
07 Aug, 2020 6 commits
- Merge branch 'pip_update' into 'main' · 3d1cbecf
  Mohammad Shoeybi authored Aug 07, 2020
```
pip update

See merge request ADLR/megatron-lm!112
```
  3d1cbecf
- pip update · 8ad2af6d
  Evelina authored Aug 07, 2020
```
Signed-off-by: Evelina <ebakhturina@nvidia.com>
```
  8ad2af6d
- Merge branch 'optimize_initialization' into 'main' · 5ba627ec
  Jared Casper authored Aug 07, 2020
```
added gpu initialization and option to avoid master values for initialization

See merge request ADLR/megatron-lm!105
```
  5ba627ec
- Merge branch 'parallel-nemo' into 'main' · 0a74547c
  Jared Casper authored Aug 07, 2020
```
Changes for NeMo/lightning compatibility

See merge request ADLR/megatron-lm!104
```
  0a74547c
- comment fix · 4e8ace9b
  Boris Fomitchev authored Aug 06, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  4e8ace9b
- fixed typo in comment · 39c80331
  Boris Fomitchev authored Aug 06, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  39c80331
06 Aug, 2020 4 commits
- Addressing code review comments · c632d205
  Boris Fomitchev authored Aug 06, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  c632d205
- Removed assert in case of external ddp, exported initialize_megatron() · be842037
  Boris Fomitchev authored Aug 06, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  be842037
- Merge remote-tracking branch 'upstream/master' into parallel-nemo · 323c124d
  Boris Fomitchev authored Aug 05, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  323c124d
- Implementing lazy parallel initialization · 5c04ceb3
  Boris Fomitchev authored Aug 05, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  5c04ceb3
04 Aug, 2020 2 commits
- Merge branch 'pip_update' into 'master' · bdc52c19
  Mohammad Shoeybi authored Aug 04, 2020
```
pip version update

See merge request ADLR/megatron-lm!107
```
  bdc52c19
- pip version update · 3b9ea79c
  Evelina authored Aug 04, 2020
```
Signed-off-by: Evelina <ebakhturina@nvidia.com>
```
  3b9ea79c
03 Aug, 2020 1 commit
- added gpu initialization and option to avoid master values · 0403b808
  mohammad authored Aug 03, 2020
  
  0403b808
31 Jul, 2020 4 commits
- Merge remote-tracking branch 'upstream/master' into parallel-nemo · cf9bdadc
  Boris Fomitchev authored Jul 31, 2020
  
  cf9bdadc
- Merge branch 'onnx-erf' into 'master' · 61f50c45
  Jared Casper authored Jul 31, 2020
```
ONNX export fix, including one for gelu()

See merge request ADLR/megatron-lm!101
```
  61f50c45
- Merge branch 'indexer-merge' into 'master' · 4ac91436
  Jared Casper authored Jul 31, 2020
```
REALM indexer and index data structures

See merge request ADLR/megatron-lm!96
```
  4ac91436
- Code review comments - changing parallel test condition · 35bea728
  Boris Fomitchev authored Jul 30, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  35bea728
30 Jul, 2020 1 commit
- Remove spurious imports · 33a666d4
  Neel Kant authored Jul 30, 2020
  
  33a666d4