Commits · 1d4e8760cbf6cdb00c5f6ba2d676f8fb5a87fa7a · OpenDAS / Megatron-LM

19 Dec, 2020 17 commits
- Fix text generation without recompute · 1d4e8760
  Jared Casper authored Dec 10, 2020
  
  1d4e8760
- Nicer error messages for deprecated arguments · 2623551d
  Jared Casper authored Dec 10, 2020
  
  2623551d
- Change lr-warmup-percent to lr-warmup-fraction · 9321d5c6
  Jared Casper authored Dec 10, 2020
  
  9321d5c6
- Add implementation for pipelined zeroshot GPT-2 evaluation · 0c151638
  Jared Casper authored Dec 09, 2020
  
  0c151638
- Work batch-size name changes into task code · 3afcba6e
  Jared Casper authored Dec 09, 2020
  
  3afcba6e
- Initial implementation of pipelined text generation · 5c45db4a
  Jared Casper authored Dec 09, 2020
  
  5c45db4a
- Add pipelining to GLUE and RACE tasks · caa9dca5
  Jared Casper authored Nov 30, 2020
  
  caa9dca5
- Better memory tracking across pipeline-parallel ranks · 3574b8e6
  Deepak Narayanan authored Dec 06, 2020
  
  3574b8e6
- Address Jared's comments · 00ac56ab
  mohammad authored Dec 09, 2020
  
  00ac56ab
- Sample based learning rate computation · 22ab91bb
  mohammad authored Dec 08, 2020
  
  22ab91bb
- Minor fixes for batch size rampup · 6a68502d
  mohammad authored Dec 08, 2020
  
  6a68502d
- Support for ramping up the batch size · de0b70a0
  mohammad authored Dec 08, 2020
  
  de0b70a0
- Minor refactoring · c30ba0f7
  mohammad authored Dec 08, 2020
  
  c30ba0f7
- Add constant num micro-batches calculator · feecd5d9
  mohammad authored Dec 07, 2020
  
  feecd5d9
- Add micro-batch size calculator · 6ea23928
  mohammad authored Dec 06, 2020
  
  6ea23928
- Rename --batch-size to --micro-batch-size and drop in-minibatch from... · 9019bbf4
  mohammad authored Dec 06, 2020
```
Rename --batch-size to --micro-batch-size and drop in-minibatch from --num-micro-batches-in-minibatch
```
  9019bbf4
- Make an eval iteration the same number of samples as a training iteration · a84a5fa0
  Jared Casper authored Dec 03, 2020
  
  a84a5fa0
03 Dec, 2020 4 commits
- Merge branch 'main' into pipeline_parallel_merge · 2cf1d6d0
  Jared Casper authored Dec 03, 2020
  
  2cf1d6d0
- Merge branch 'consumed_tokens_restart_fix' into 'main' · 3aacd955
  Jared Casper authored Dec 03, 2020
```
found a bug in consumed tokens initialization

See merge request ADLR/megatron-lm!182
```
  3aacd955
- found a bug in consumed tokens initialization · e2a4d426
  mohammad authored Dec 02, 2020
  
  e2a4d426
- Merge branch 'main' into pipeline_parallel_main · 91d4a605
  Jared Casper authored Dec 02, 2020
  
  91d4a605
02 Dec, 2020 8 commits
- Merge branch 'megatron_sampler' into 'main' · 75bd9b54
  Jared Casper authored Dec 02, 2020
```
Simplified sampler (will be needed later for batch size increase) and removed deprecated data stuff

See merge request ADLR/megatron-lm!177
```
  75bd9b54
- Merge branch 'blendable_dataset' into 'megatron_sampler' · fac6718a
  Jared Casper authored Dec 02, 2020
```
Blendable dataset

See merge request ADLR/megatron-lm!178
```
  fac6718a
- Merge branch 'refactor_learning_rate' into 'blendable_dataset' · 1eda0a17
  Jared Casper authored Dec 02, 2020
```
Refactor learning rate so it is easier to make learning rate based on consumed samples

See merge request ADLR/megatron-lm!179
```
  1eda0a17
- addressed Jareds comments · fa80af26
  mohammad authored Dec 02, 2020
  
  fa80af26
- Merge branch 'blendable_dataset' into refactor_learning_rate · 45504541
  mohammad authored Dec 02, 2020
  
  45504541
- addressed Jareds comments · 98989693
  mohammad authored Dec 02, 2020
  
  98989693
- Merge branch 'megatron_sampler' into blendable_dataset · bc56e4a5
  mohammad authored Dec 02, 2020
  
  bc56e4a5
- addrressed jareds comments · cebd3b8b
  mohammad authored Dec 02, 2020
  
  cebd3b8b
30 Nov, 2020 2 commits
- refactored learning rate scheduler so addition of variable batch size is easier · ff12df6b
  mohammad authored Nov 29, 2020
  
  ff12df6b
- added refactored learning rate · 16193619
  mohammad authored Nov 29, 2020
  
  16193619
29 Nov, 2020 3 commits
- implemented blending datasets · 65290033
  mohammad authored Nov 28, 2020
  
  65290033
- Merge branch 'megatron_sampler' into blendable_dataset · 9a0808c9
  mohammad authored Nov 28, 2020
  
  9a0808c9
- added blendable dataset · d3bb1a06
  mohammad authored Nov 28, 2020
  
  d3bb1a06
28 Nov, 2020 1 commit
- added consumed tokens to checkpoints and some refactoring · f0a445fa
  mohammad authored Nov 27, 2020
  
  f0a445fa
26 Nov, 2020 1 commit
- simplified sampler · 4311b695
  mohammad authored Nov 25, 2020
  
  4311b695
19 Nov, 2020 1 commit
- Merge branch 'main' into pipeline_parallel_main · 63c340ec
  Jared Casper authored Nov 19, 2020
  
  63c340ec
18 Nov, 2020 3 commits
- Merge branch 'update-norm' into 'main' · ea81d62f
  Mohammad Shoeybi authored Nov 17, 2020
```
Replace deprecated torch.norm with torch.linalg.norm.

See merge request ADLR/megatron-lm!175
```
  ea81d62f
- Merge branch 'community-fixes' into 'main' · ac837a4e
  Mohammad Shoeybi authored Nov 17, 2020
```
Community fixes

See merge request ADLR/megatron-lm!176
```
  ac837a4e
- Merge branch 'fix/help-title-dist' of https://github.com/lazykyama/Megatron-LM into community-fixes · 356f8771
  Jared Casper authored Nov 17, 2020
  
  356f8771