Commits · a4f4188210f316df90a4c0cef2edc3d77aae164b · wxj / Megatron-LM

"official/projects/s3d/modeling/inception_utils.py" did not exist on "8c3a1ef3796dabeee371ca33011fa600fb061d67"

18 Feb, 2022 4 commits
- copy model params to main params at very start · a4f41882
  Lawrence McAfee authored Feb 18, 2022
  
  a4f41882
- some cleanup · ae6cd9b5
  Lawrence McAfee authored Feb 18, 2022
  
  ae6cd9b5
- modularized reduce_gradients, gather params; trainin runs, but loss==nan · e5db0fda
  Lawrence McAfee authored Feb 18, 2022
  
  e5db0fda
- fixed clip_grad_norm bug · a7782b21
  Lawrence McAfee authored Feb 18, 2022
  
  a7782b21
16 Feb, 2022 1 commit
- small changes · d1f5776a
  Lawrence McAfee authored Feb 15, 2022
  
  d1f5776a
15 Feb, 2022 7 commits
- reduce scatter working · ff4c4273
  Lawrence McAfee authored Feb 15, 2022
  
  ff4c4273
- added 'all' world shards to gbuf map · bf64c85c
  Lawrence McAfee authored Feb 15, 2022
  
  bf64c85c
- updated zero_grad · eaa0c1df
  Lawrence McAfee authored Feb 15, 2022
  
  eaa0c1df
- allocated main params/grads · 33626179
  Lawrence McAfee authored Feb 15, 2022
  
  33626179
- created optimizer group shards · 525a8351
  Lawrence McAfee authored Feb 15, 2022
  
  525a8351
- modularized shard indexing · f498a6fe
  Lawrence McAfee authored Feb 15, 2022
  
  f498a6fe
- wip; switching to grad-buffer-centric design · cb6f96b6
  Lawrence McAfee authored Feb 15, 2022
  
  cb6f96b6
14 Feb, 2022 8 commits
- todo; align shards with model's contiguous buffer · a3f3c3ad
  Lawrence McAfee authored Feb 14, 2022
  
  a3f3c3ad
- copying model grad slices to main grad · 3f0bc681
  Lawrence McAfee authored Feb 14, 2022
  
  3f0bc681
- fix zero_grad; set_to_none = False · 6875dff5
  Lawrence McAfee authored Feb 14, 2022
  
  6875dff5
- tweaked slice index naming convention · 1215c420
  Lawrence McAfee authored Feb 14, 2022
  
  1215c420
- map param to originating virtual model; eventually move this to constructor · c5f93269
  Lawrence McAfee authored Feb 14, 2022
  
  c5f93269
- included original param index in map · 3ded2425
  Lawrence McAfee authored Feb 14, 2022
  
  3ded2425
- built local shard param index map · a74e245c
  Lawrence McAfee authored Feb 14, 2022
  
  a74e245c
- collect param offsets for contiguous grad buffer · f7232502
  Lawrence McAfee authored Feb 14, 2022
  
  f7232502
11 Feb, 2022 2 commits
- bit more progress · 5706ba42
  Lawrence McAfee authored Feb 11, 2022
  
  5706ba42
- studied float16 optimizer; more updates · f48e1f29
  Lawrence McAfee authored Feb 11, 2022
  
  f48e1f29
10 Feb, 2022 2 commits
- more work on Float16DistributedOptimizer · 49cca4d9
  Lawrence McAfee authored Feb 10, 2022
  
  49cca4d9
- working on Float16DistributedOptimizer · 329fe582
  Lawrence McAfee authored Feb 10, 2022
  
  329fe582
09 Feb, 2022 1 commit
- feb 9 alpha · 7dc8c475
  Lawrence McAfee authored Feb 09, 2022
  
  7dc8c475
29 Jan, 2022 4 commits
- Merge branch 'vision-merge' into 'main' · e724785f
  Jared Casper authored Jan 28, 2022
```
second phase of vision code merge

See merge request ADLR/megatron-lm!381
```
  e724785f
- typo fix · 2b628f96
  Vijay Korthikanti authored Jan 28, 2022
  
  2b628f96
- Merge branch 'github-pr' into 'main' · e156d2fe
  Jared Casper authored Jan 28, 2022
```
Combination of several github PRs

See merge request ADLR/megatron-lm!383
```
  e156d2fe
- Revert incorrect fix. · cd499559
  Jared Casper authored Jan 28, 2022
  
  cd499559
28 Jan, 2022 9 commits
- Merge branch 'patch-1' of https://github.com/vycezhong/Megatron-LM into github-pr · 2a34e0ec
  Jared Casper authored Jan 28, 2022
  
  2a34e0ec
- Merge branch 'main' of https://github.com/satpalsr/Megatron-LM into github-pr · 34f55429
  Jared Casper authored Jan 28, 2022
  
  34f55429
- Merge branch 'patch-1' of https://github.com/jamesr66a/Megatron-LM into github-pr · adebe364
  Jared Casper authored Jan 28, 2022
  
  adebe364
- Merge branch 'patch-2' of https://github.com/kvtoraman/Megatron-LM into github-pr · 20f6169f
  Jared Casper authored Jan 28, 2022
  
  20f6169f
- Merge branch 'patch-1' of https://github.com/rajeshkppt/Megatron-LM into github-pr · 0747e8e5
  Jared Casper authored Jan 28, 2022
  
  0747e8e5
- Merge branch 'fix' of https://github.com/singleheart/Megatron-LM into github-pr · 9882fb3f
  Jared Casper authored Jan 28, 2022
  
  9882fb3f
- Merge branch 'patch-1' of https://github.com/stas00/Megatron-LM into github-pr · 4a62d582
  Jared Casper authored Jan 28, 2022
  
  4a62d582
- more naming cleanup · 641408f5
  Vijay Korthikanti authored Jan 28, 2022
  
  641408f5
- changing class name AnnealingLR to OptimizerParamScheduler · 04ecc834
  Vijay Korthikanti authored Jan 28, 2022
  
  04ecc834
27 Jan, 2022 2 commits
- address review comments · 53931b8b
  Vijay Korthikanti authored Jan 27, 2022
  
  53931b8b
- Merge branch 'lmcafee/distrib-chkpt-fix-v2' into 'main' · fd5469aa
  Jared Casper authored Jan 26, 2022
```
Distributed checkpointing memory fix

See merge request ADLR/megatron-lm!379
```
  fd5469aa