Commits · 42d21122043bfb860e9c73da2196b17f87826420 · OpenDAS / Megatron-LM

09 Oct, 2020 3 commits
- Merge branch 'build_fix' into 'main' · 42d21122
  Mohammad Shoeybi authored Oct 09, 2020
```
Create build directory for fused_kernels before building.

See merge request ADLR/megatron-lm!149
```
  42d21122
- Merge branch 'vijay/diff_query_key_lengths' into 'main' · 54282071
  Mohammad Shoeybi authored Oct 08, 2020
```
support for different query key sequence lengths

See merge request ADLR/megatron-lm!151
```
  54282071
- support for different query key sequence lengths · d763f54a
  Vijay Korthikanti authored Oct 08, 2020
  
  d763f54a
08 Oct, 2020 1 commit
- Create build directory for fused_kernels before building. · 54ded172
  Jared Casper authored Oct 07, 2020
  
  54ded172
06 Oct, 2020 2 commits
- Merge branch 'vijay/fused_kernel_compiliation' into 'main' · 56e16cba
  Jared Casper authored Oct 05, 2020
```
Avoid non deterministic arch order in compilation

See merge request ADLR/megatron-lm!146
```
  56e16cba
- build directory · 3dcd7dc0
  Vijay Korthikanti authored Oct 05, 2020
  
  3dcd7dc0
05 Oct, 2020 2 commits
- build_directory argument · a7f4e460
  Vijay Korthikanti authored Oct 05, 2020
  
  a7f4e460
- avoid non deterministic arch order in compilation · 37d77ca0
  Vijay Korthikanti authored Oct 05, 2020
  
  37d77ca0
02 Oct, 2020 5 commits
- Merge branch 'pip_1.1.5' into 'main' · d80433e1
  Mohammad Shoeybi authored Oct 02, 2020
```
pip 1.1.5

See merge request ADLR/megatron-lm!144
```
  d80433e1
- pip 1.1.5 · f9387ec3
  Evelina authored Oct 02, 2020
```
Signed-off-by: Evelina <ebakhturina@nvidia.com>
```
  f9387ec3
- Merge branch 'pip_1.1.4' into 'main' · b7dac89a
  Mohammad Shoeybi authored Oct 01, 2020
```
1.1.4. pip version

See merge request ADLR/megatron-lm!120
```
  b7dac89a
- Merge branch 'mpatwary/main_text_gen' into 'main' · b1870045
  Mohammad Shoeybi authored Oct 01, 2020
```
Fixed uninit communication gruop

See merge request ADLR/megatron-lm!142
```
  b1870045
- Fixed uninit communication gruop · 057a1fab
  Mostofa Patwary authored Oct 01, 2020
  
  057a1fab
01 Oct, 2020 2 commits
- Merge branch 'check_mpu_init' into 'main' · 10c828ed
  Jared Casper authored Oct 01, 2020
```
fix for nemo: do not initialize mpu if it is already initialized

See merge request ADLR/megatron-lm!141
```
  10c828ed
- fix for nemo: do not initialize mpu if it is already initialized · 6e433055
  mohammad authored Oct 01, 2020
  
  6e433055
30 Sep, 2020 3 commits
- Merge branch 'vijay/weight_mapping' into 'main' · c63906a6
  Mohammad Shoeybi authored Sep 29, 2020
```
Vijay/weight mapping

See merge request ADLR/megatron-lm!138
```
  c63906a6
- address review comments · 64e45f29
  Vijay Korthikanti authored Sep 29, 2020
  
  64e45f29
- checkpoint versioning · a8d47812
  Vijay Korthikanti authored Sep 29, 2020
  
  a8d47812
29 Sep, 2020 5 commits
- Merge branch 'main_megatron_cntrl' into 'main' · 577b4657
  Mohammad Shoeybi authored Sep 29, 2020
```
Text generation fixes

See merge request ADLR/megatron-lm!137
```
  577b4657
- bug fix · 7d4ad51e
  Vijay Korthikanti authored Sep 29, 2020
  
  7d4ad51e
- fix of few uninitialized varibles · 826d2452
  Mostofa Patwary authored Sep 29, 2020
  
  826d2452
- support for old checkpoint format · 37ae6646
  Vijay Korthikanti authored Sep 28, 2020
  
  37ae6646
- Text generation bugs · 1f3f834c
  Mostofa Patwary authored Sep 28, 2020
  
  1f3f834c
25 Sep, 2020 3 commits
- Merge branch 'log_report_fix' into 'main' · 4bf923d5
  Jared Casper authored Sep 25, 2020
```
fixed logging bug

See merge request ADLR/megatron-lm!134
```
  4bf923d5
- fixed the interation count · 2db01991
  mohammad authored Sep 24, 2020
  
  2db01991
- Merge branch 'main' into log_report_fix · 9787e6c9
  mohammad authored Sep 24, 2020
  
  9787e6c9
24 Sep, 2020 2 commits
- Merge branch 'vijay/softmax_fusion' into 'main' · 2bd27e20
  Jared Casper authored Sep 24, 2020
```
scaling-generalized masking-softmax fusion kernel

See merge request ADLR/megatron-lm!131
```
  2bd27e20
- nan values report added · abd7636e
  mohammad authored Sep 23, 2020
  
  abd7636e
22 Sep, 2020 1 commit
- fixed logging bug · a74f8d9c
  mohammad authored Sep 22, 2020
  
  a74f8d9c
18 Sep, 2020 1 commit
- scaling-generalized masking-softmax fusion kernel · d73def35
  root authored Sep 18, 2020
  
  d73def35
11 Sep, 2020 4 commits
- Merge branch 'readme_update' into 'main' · 19301985
  Jared Casper authored Sep 11, 2020
```
Update scaling numbers in README and other small tweaks.

See merge request ADLR/megatron-lm!130
```
  19301985
- Update scaling numbers in README and other small tweaks. · 52819194
  Jared Casper authored Sep 11, 2020
  
  52819194
- Merge branch 'vijay/softmax_fusion' into 'main' · 91ee60df
  Jared Casper authored Sep 11, 2020
```
Various speed optimizations.

See merge request ADLR/megatron-lm!124
```
  91ee60df
- Various speed optimizations. · 51a2e6b0
  Vijay Korthikanti authored Sep 11, 2020
  
  51a2e6b0
03 Sep, 2020 3 commits
- Merge branch 'dist_act_chkpt' into 'main' · 12518332
  Jared Casper authored Sep 03, 2020
```
added splitting checkpointed activations across model parallel partitions

See merge request ADLR/megatron-lm!121
```
  12518332
- Merge branch 'fix_cpu_init' into 'main' · 930ec4a2
  Jared Casper authored Sep 03, 2020
```
Fix cpu initialization for mp=1 and fp16

See merge request ADLR/megatron-lm!123
```
  930ec4a2
- debuged the cpu init for mp=1 · 04cd82bd
  mohammad authored Sep 03, 2020
  
  04cd82bd
02 Sep, 2020 2 commits
- addressed Jareds comments · 5d29769c
  mohammad authored Sep 02, 2020
  
  5d29769c
- added splitting checkpointed activations · d6c4248b
  mohammad authored Sep 02, 2020
  
  d6c4248b
01 Sep, 2020 1 commit
- 1.1.4. pip version · e8c47e76
  Evelina authored Sep 01, 2020
```
Signed-off-by: Evelina <ebakhturina@nvidia.com>
```
  e8c47e76