Commits · a74e245cfca7b6a21dce0807eedede5318b0456e · OpenDAS / Megatron-LM

"tests/vscode:/vscode.git/clone" did not exist on "f7813d6d3321fbef5dfe04f4294ed913c8c784e6"

14 Feb, 2022 2 commits
- built local shard param index map · a74e245c
  Lawrence McAfee authored Feb 14, 2022
  
  a74e245c
- collect param offsets for contiguous grad buffer · f7232502
  Lawrence McAfee authored Feb 14, 2022
  
  f7232502
11 Feb, 2022 2 commits
- bit more progress · 5706ba42
  Lawrence McAfee authored Feb 11, 2022
  
  5706ba42
- studied float16 optimizer; more updates · f48e1f29
  Lawrence McAfee authored Feb 11, 2022
  
  f48e1f29
10 Feb, 2022 2 commits
- more work on Float16DistributedOptimizer · 49cca4d9
  Lawrence McAfee authored Feb 10, 2022
  
  49cca4d9
- working on Float16DistributedOptimizer · 329fe582
  Lawrence McAfee authored Feb 10, 2022
  
  329fe582
09 Feb, 2022 1 commit
- feb 9 alpha · 7dc8c475
  Lawrence McAfee authored Feb 09, 2022
  
  7dc8c475
29 Jan, 2022 4 commits
- Merge branch 'vision-merge' into 'main' · e724785f
  Jared Casper authored Jan 28, 2022
```
second phase of vision code merge

See merge request ADLR/megatron-lm!381
```
  e724785f
- typo fix · 2b628f96
  Vijay Korthikanti authored Jan 28, 2022
  
  2b628f96
- Merge branch 'github-pr' into 'main' · e156d2fe
  Jared Casper authored Jan 28, 2022
```
Combination of several github PRs

See merge request ADLR/megatron-lm!383
```
  e156d2fe
- Revert incorrect fix. · cd499559
  Jared Casper authored Jan 28, 2022
  
  cd499559
28 Jan, 2022 9 commits
- Merge branch 'patch-1' of https://github.com/vycezhong/Megatron-LM into github-pr · 2a34e0ec
  Jared Casper authored Jan 28, 2022
  
  2a34e0ec
- Merge branch 'main' of https://github.com/satpalsr/Megatron-LM into github-pr · 34f55429
  Jared Casper authored Jan 28, 2022
  
  34f55429
- Merge branch 'patch-1' of https://github.com/jamesr66a/Megatron-LM into github-pr · adebe364
  Jared Casper authored Jan 28, 2022
  
  adebe364
- Merge branch 'patch-2' of https://github.com/kvtoraman/Megatron-LM into github-pr · 20f6169f
  Jared Casper authored Jan 28, 2022
  
  20f6169f
- Merge branch 'patch-1' of https://github.com/rajeshkppt/Megatron-LM into github-pr · 0747e8e5
  Jared Casper authored Jan 28, 2022
  
  0747e8e5
- Merge branch 'fix' of https://github.com/singleheart/Megatron-LM into github-pr · 9882fb3f
  Jared Casper authored Jan 28, 2022
  
  9882fb3f
- Merge branch 'patch-1' of https://github.com/stas00/Megatron-LM into github-pr · 4a62d582
  Jared Casper authored Jan 28, 2022
  
  4a62d582
- more naming cleanup · 641408f5
  Vijay Korthikanti authored Jan 28, 2022
  
  641408f5
- changing class name AnnealingLR to OptimizerParamScheduler · 04ecc834
  Vijay Korthikanti authored Jan 28, 2022
  
  04ecc834
27 Jan, 2022 4 commits
- address review comments · 53931b8b
  Vijay Korthikanti authored Jan 27, 2022
  
  53931b8b
- Merge branch 'lmcafee/distrib-chkpt-fix-v2' into 'main' · fd5469aa
  Jared Casper authored Jan 26, 2022
```
Distributed checkpointing memory fix

See merge request ADLR/megatron-lm!379
```
  fd5469aa
- add clarification about the model parallel size column · 2d767f48
  Stas Bekman authored Jan 26, 2022
  
  2d767f48
- [README] specify explicitly which gpu and node size was used · 145d2eb9
  Stas Bekman authored Jan 26, 2022
```
The paper has this info, so proposing to copy it next to the table. 

Otherwise it's hard to guess whether you used 40GB A100s or 80GB ones (and secondary, n_gpus per node).

Thank you!
```
  145d2eb9
26 Jan, 2022 2 commits
- further clarified viewless tensor comment in transformer.py · d16e2a24
  Lawrence McAfee authored Jan 26, 2022
  
  d16e2a24
- address review comments · 8acbbe25
  Vijay Korthikanti authored Jan 26, 2022
  
  8acbbe25
25 Jan, 2022 2 commits
- Merge branch 'lmcafee/t5-free-pipe-output-true-fix' into 'main' · fc5d4c2b
  Jared Casper authored Jan 25, 2022
```
T5 fix for deallocating pipeline output tensors

See merge request ADLR/megatron-lm!380
```
  fc5d4c2b
- limit 'make_viewless_tensor()' to case of micro_batch_size == 1; added comment · 24369dd6
  Lawrence McAfee authored Jan 25, 2022
  
  24369dd6
24 Jan, 2022 1 commit
- minor fixes · 7e810e41
  Vijay Korthikanti authored Jan 24, 2022
  
  7e810e41
22 Jan, 2022 2 commits
- fix an argument · 313730e9
  Jay authored Jan 22, 2022
  
  313730e9
- second phase of vision code merge · d8c85650
  Vijay Korthikanti authored Jan 21, 2022
  
  d8c85650
20 Jan, 2022 2 commits
- Merge branch 'vision-merge' into 'main' · 539e39b6
  Jared Casper authored Jan 20, 2022
```
Phase1 merge: vit optimizations + dataset enhancements + scaled_softmax kernel

See merge request ADLR/megatron-lm!378
```
  539e39b6
- address review comments · 798b6a64
  Vijay Korthikanti authored Jan 20, 2022
  
  798b6a64
19 Jan, 2022 2 commits
- using all_gather instead of gather (nccl does not support gather) · b037a69e
  Vijay Korthikanti authored Jan 19, 2022
  
  b037a69e
- flag for data parallel random initialization · a7ee77ea
  Vijay Korthikanti authored Jan 19, 2022
  
  a7ee77ea
15 Jan, 2022 2 commits
- minor fix · fd8dd9c0
  Vijay Korthikanti authored Jan 14, 2022
  
  fd8dd9c0
- checkpointing rng_state of all data parallel ranks · 09d38884
  Vijay Korthikanti authored Jan 14, 2022
  
  09d38884
14 Jan, 2022 1 commit
- address review comments · 343dc97a
  Vijay Korthikanti authored Jan 14, 2022
  
  343dc97a
13 Jan, 2022 2 commits
- handle output_tensor == None · 7f1c5917
  Lawrence McAfee authored Jan 13, 2022
  
  7f1c5917
- more cleanup · 8fc5e323
  Lawrence McAfee authored Jan 13, 2022
  
  8fc5e323