Commits · 75bd9b54aaf37d485c4be2eab77e61fcc6748431 · OpenDAS / Megatron-LM

"vscode:/vscode.git/clone" did not exist on "451d15c44be3e2e06cf71756f2a96d5e293f610f"

02 Dec, 2020 8 commits
- Merge branch 'megatron_sampler' into 'main' · 75bd9b54
  Jared Casper authored Dec 02, 2020
```
Simplified sampler (will be needed later for batch size increase) and removed deprecated data stuff

See merge request ADLR/megatron-lm!177
```
  75bd9b54
- Merge branch 'blendable_dataset' into 'megatron_sampler' · fac6718a
  Jared Casper authored Dec 02, 2020
```
Blendable dataset

See merge request ADLR/megatron-lm!178
```
  fac6718a
- Merge branch 'refactor_learning_rate' into 'blendable_dataset' · 1eda0a17
  Jared Casper authored Dec 02, 2020
```
Refactor learning rate so it is easier to make learning rate based on consumed samples

See merge request ADLR/megatron-lm!179
```
  1eda0a17
- addressed Jareds comments · fa80af26
  mohammad authored Dec 02, 2020
  
  fa80af26
- Merge branch 'blendable_dataset' into refactor_learning_rate · 45504541
  mohammad authored Dec 02, 2020
  
  45504541
- addressed Jareds comments · 98989693
  mohammad authored Dec 02, 2020
  
  98989693
- Merge branch 'megatron_sampler' into blendable_dataset · bc56e4a5
  mohammad authored Dec 02, 2020
  
  bc56e4a5
- addrressed jareds comments · cebd3b8b
  mohammad authored Dec 02, 2020
  
  cebd3b8b
30 Nov, 2020 2 commits
- refactored learning rate scheduler so addition of variable batch size is easier · ff12df6b
  mohammad authored Nov 29, 2020
  
  ff12df6b
- added refactored learning rate · 16193619
  mohammad authored Nov 29, 2020
  
  16193619
29 Nov, 2020 3 commits
- implemented blending datasets · 65290033
  mohammad authored Nov 28, 2020
  
  65290033
- Merge branch 'megatron_sampler' into blendable_dataset · 9a0808c9
  mohammad authored Nov 28, 2020
  
  9a0808c9
- added blendable dataset · d3bb1a06
  mohammad authored Nov 28, 2020
  
  d3bb1a06
28 Nov, 2020 1 commit
- added consumed tokens to checkpoints and some refactoring · f0a445fa
  mohammad authored Nov 27, 2020
  
  f0a445fa
26 Nov, 2020 1 commit
- simplified sampler · 4311b695
  mohammad authored Nov 25, 2020
  
  4311b695
18 Nov, 2020 9 commits
- Merge branch 'update-norm' into 'main' · ea81d62f
  Mohammad Shoeybi authored Nov 17, 2020
```
Replace deprecated torch.norm with torch.linalg.norm.

See merge request ADLR/megatron-lm!175
```
  ea81d62f
- Merge branch 'community-fixes' into 'main' · ac837a4e
  Mohammad Shoeybi authored Nov 17, 2020
```
Community fixes

See merge request ADLR/megatron-lm!176
```
  ac837a4e
- Merge branch 'fix/help-title-dist' of https://github.com/lazykyama/Megatron-LM into community-fixes · 356f8771
  Jared Casper authored Nov 17, 2020
  
  356f8771
- Merge branch 'patch-1' of https://github.com/sublee/Megatron-LM into community-fixes · 24ccd8aa
  Jared Casper authored Nov 17, 2020
  
  24ccd8aa
- Merge branch 'patch-2' of https://github.com/nakosung/Megatron-LM into community-fixes · cec3dbe6
  Jared Casper authored Nov 17, 2020
  
  cec3dbe6
- Merge branch 'patch-1' of https://github.com/nakosung/Megatron-LM into community-fixes · 8d0da202
  Jared Casper authored Nov 17, 2020
  
  8d0da202
- Merge branch 'patch-1' of https://github.com/dweekly/Megatron-LM into community-fixes · 5e934043
  Jared Casper authored Nov 17, 2020
  
  5e934043
- Merge branch 'patch-1' of https://github.com/akhileshgotmare/Megatron-LM into community-fixes · 642c644a
  Jared Casper authored Nov 17, 2020
  
  642c644a
- Replace deprecated torch.norm with torch.linalg.norm. · 17035d6c
  Jared Casper authored Nov 17, 2020
  
  17035d6c
17 Nov, 2020 2 commits
- Merge branch 'finetune_fix' into 'main' · e7c7a78f
  Mohammad Shoeybi authored Nov 16, 2020
```
Update code used for finetuning to latest API.

See merge request ADLR/megatron-lm!174
```
  e7c7a78f
- Update code used for finetuning to latest API. · b219ff00
  Jared Casper authored Nov 16, 2020
  
  b219ff00
03 Nov, 2020 2 commits
- Merge branch 'fix_logging' into 'main' · b4b0d739
  Deepak Narayanan authored Nov 03, 2020
```
fixed loss average when all but one value is skipped

See merge request ADLR/megatron-lm!164
```
  b4b0d739
- fixed loss average when all but one value is skipped · 664cd28b
  mohammad authored Nov 03, 2020
  
  664cd28b
20 Oct, 2020 1 commit
- Change distributed argument group title to appropriate one. · c147a3e8
  Kazuhiro Yamasaki authored Oct 20, 2020
  
  c147a3e8
14 Oct, 2020 6 commits
- Merge branch 'main_evaluate_wiki' into 'main' · 79888e16
  Mohammad Shoeybi authored Oct 14, 2020
```
fixed wiki evaluation

See merge request ADLR/megatron-lm!157
```
  79888e16
- fixed wiki evaluation issue · ef2adb5d
  Mostofa Patwary authored Oct 14, 2020
  
  ef2adb5d
- fixed wiki evaluation issue · 38c45de7
  Mostofa Patwary authored Oct 14, 2020
  
  38c45de7
- Merge branch 'main_beta' into 'main' · 64cf3d98
  Mohammad Shoeybi authored Oct 13, 2020
```
Adam betas and eps

See merge request ADLR/megatron-lm!156
```
  64cf3d98
- added adam betas and eps as arguments · c55e154f
  Mostofa Patwary authored Oct 13, 2020
  
  c55e154f
- added adam betas and eps as arguments · 48269d8d
  Mostofa Patwary authored Oct 13, 2020
  
  48269d8d
13 Oct, 2020 5 commits
- Merge branch 'vijay/suppress_ninja_output' into 'main' · 5753e8f1
  Mohammad Shoeybi authored Oct 12, 2020
```
suppress kernel loading output

See merge request ADLR/megatron-lm!155
```
  5753e8f1
- suppress kernel loading output · 8fca9b49
  Vijay Korthikanti authored Oct 12, 2020
  
  8fca9b49
- Merge branch 'vijay/ordering_perf_fix' into 'main' · 538f0e05
  Mohammad Shoeybi authored Oct 12, 2020
```
reordering perf fix

See merge request ADLR/megatron-lm!154
```
  538f0e05
- typo fix · 01dffdb5
  Vijay Korthikanti authored Oct 12, 2020
  
  01dffdb5
- address review comments · 0bc75448
  Vijay Korthikanti authored Oct 12, 2020
  
  0bc75448