Commits · a74f8d9c5d5b57147293c08fc2918b2172fbbfb3 · OpenDAS / Megatron-LM

22 Sep, 2020 1 commit
- fixed logging bug · a74f8d9c
  mohammad authored Sep 22, 2020
  
  a74f8d9c
11 Sep, 2020 4 commits
- Merge branch 'readme_update' into 'main' · 19301985
  Jared Casper authored Sep 11, 2020
```
Update scaling numbers in README and other small tweaks.

See merge request ADLR/megatron-lm!130
```
  19301985
- Update scaling numbers in README and other small tweaks. · 52819194
  Jared Casper authored Sep 11, 2020
  
  52819194
- Merge branch 'vijay/softmax_fusion' into 'main' · 91ee60df
  Jared Casper authored Sep 11, 2020
```
Various speed optimizations.

See merge request ADLR/megatron-lm!124
```
  91ee60df
- Various speed optimizations. · 51a2e6b0
  Vijay Korthikanti authored Sep 11, 2020
  
  51a2e6b0
03 Sep, 2020 3 commits
- Merge branch 'dist_act_chkpt' into 'main' · 12518332
  Jared Casper authored Sep 03, 2020
```
added splitting checkpointed activations across model parallel partitions

See merge request ADLR/megatron-lm!121
```
  12518332
- Merge branch 'fix_cpu_init' into 'main' · 930ec4a2
  Jared Casper authored Sep 03, 2020
```
Fix cpu initialization for mp=1 and fp16

See merge request ADLR/megatron-lm!123
```
  930ec4a2
- debuged the cpu init for mp=1 · 04cd82bd
  mohammad authored Sep 03, 2020
  
  04cd82bd
02 Sep, 2020 2 commits
- addressed Jareds comments · 5d29769c
  mohammad authored Sep 02, 2020
  
  5d29769c
- added splitting checkpointed activations · d6c4248b
  mohammad authored Sep 02, 2020
  
  d6c4248b
01 Sep, 2020 1 commit

Merge branch 'onnx-safe' into 'main' · 42d2be06

Mohammad Shoeybi authored Sep 01, 2020

--onnx-safe made overrideable, -lazy_mpu_init act as _USE_CPU_INITIALIZATION

See merge request ADLR/megatron-lm!117

42d2be06

28 Aug, 2020 1 commit
- Added documentation · b6b48a32
  Boris Fomitchev authored Aug 27, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  b6b48a32
27 Aug, 2020 1 commit
- Addressing code review comments · 1eafa861
  Boris Fomitchev authored Aug 26, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  1eafa861
26 Aug, 2020 1 commit
- Making --lazy_mpu_init act as _USE_CPU_INITIALIZATION · e10760ea
  Boris Fomitchev authored Aug 26, 2020
  
  e10760ea
25 Aug, 2020 1 commit
- Making onnx-safe option overrideable · d4bb6cb0
  Boris Fomitchev authored Aug 25, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  d4bb6cb0
07 Aug, 2020 6 commits
- Merge branch 'pip_update' into 'main' · 3d1cbecf
  Mohammad Shoeybi authored Aug 07, 2020
```
pip update

See merge request ADLR/megatron-lm!112
```
  3d1cbecf
- pip update · 8ad2af6d
  Evelina authored Aug 07, 2020
```
Signed-off-by: Evelina <ebakhturina@nvidia.com>
```
  8ad2af6d
- Merge branch 'optimize_initialization' into 'main' · 5ba627ec
  Jared Casper authored Aug 07, 2020
```
added gpu initialization and option to avoid master values for initialization

See merge request ADLR/megatron-lm!105
```
  5ba627ec
- Merge branch 'parallel-nemo' into 'main' · 0a74547c
  Jared Casper authored Aug 07, 2020
```
Changes for NeMo/lightning compatibility

See merge request ADLR/megatron-lm!104
```
  0a74547c
- comment fix · 4e8ace9b
  Boris Fomitchev authored Aug 06, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  4e8ace9b
- fixed typo in comment · 39c80331
  Boris Fomitchev authored Aug 06, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  39c80331
06 Aug, 2020 4 commits
- Addressing code review comments · c632d205
  Boris Fomitchev authored Aug 06, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  c632d205
- Removed assert in case of external ddp, exported initialize_megatron() · be842037
  Boris Fomitchev authored Aug 06, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  be842037
- Merge remote-tracking branch 'upstream/master' into parallel-nemo · 323c124d
  Boris Fomitchev authored Aug 05, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  323c124d
- Implementing lazy parallel initialization · 5c04ceb3
  Boris Fomitchev authored Aug 05, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  5c04ceb3
04 Aug, 2020 2 commits
- Merge branch 'pip_update' into 'master' · bdc52c19
  Mohammad Shoeybi authored Aug 04, 2020
```
pip version update

See merge request ADLR/megatron-lm!107
```
  bdc52c19
- pip version update · 3b9ea79c
  Evelina authored Aug 04, 2020
```
Signed-off-by: Evelina <ebakhturina@nvidia.com>
```
  3b9ea79c
03 Aug, 2020 1 commit
- added gpu initialization and option to avoid master values · 0403b808
  mohammad authored Aug 03, 2020
  
  0403b808
31 Jul, 2020 4 commits
- Merge remote-tracking branch 'upstream/master' into parallel-nemo · cf9bdadc
  Boris Fomitchev authored Jul 31, 2020
  
  cf9bdadc
- Merge branch 'onnx-erf' into 'master' · 61f50c45
  Jared Casper authored Jul 31, 2020
```
ONNX export fix, including one for gelu()

See merge request ADLR/megatron-lm!101
```
  61f50c45
- Merge branch 'indexer-merge' into 'master' · 4ac91436
  Jared Casper authored Jul 31, 2020
```
REALM indexer and index data structures

See merge request ADLR/megatron-lm!96
```
  4ac91436
- Code review comments - changing parallel test condition · 35bea728
  Boris Fomitchev authored Jul 30, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  35bea728
30 Jul, 2020 2 commits
- Remove spurious imports · 33a666d4
  Neel Kant authored Jul 30, 2020
  
  33a666d4
- Changes for NeMo/lightning compatibility · 417c7f6a
  Boris Fomitchev authored Jul 30, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  417c7f6a
29 Jul, 2020 3 commits
- Merge remote-tracking branch 'upstream/master' into onnx-erf · 84a5997a
  Boris Fomitchev authored Jul 29, 2020
  
  84a5997a
- Restored @script for openai_gelu · bb7c638f
  Boris Fomitchev authored Jul 29, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  bb7c638f
- Merge branch 'apex-fix2' into 'master' · 07ebf714
  Jared Casper authored Jul 29, 2020
```
Complete fix for APEX absence in NeMo

See merge request ADLR/megatron-lm!103
```
  07ebf714
28 Jul, 2020 1 commit
- Replacing --erf-gelu option with explicit --onnx-safe option · fe794c5a
  Boris Fomitchev authored Jul 27, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  fe794c5a
26 Jul, 2020 2 commits
- Add additional assertion on Indexer to test correctness, and limit verbosity in other classes · eaa5d877
  Neel Kant authored Jul 25, 2020
  
  eaa5d877
- fixing copy-paste · 7917774a
  Boris Fomitchev authored Jul 25, 2020
  
  7917774a