Commits · d0925652796682d4ff62e7ad99221fe749f5764d · OpenDAS / Megatron-LM

25 Mar, 2022 2 commits
- debugging fp32 training. · d0925652
  Lawrence McAfee authored Mar 25, 2022
  
  d0925652
- working: checkpoint save/load. · 9f64f5f4
  Lawrence McAfee authored Mar 25, 2022
  
  9f64f5f4
23 Mar, 2022 3 commits
- renamed timers. · 2706edc4
  Lawrence McAfee authored Mar 23, 2022
  
  2706edc4
- added timers for reduce-grads/gather-params. · cad2c80a
  Lawrence McAfee authored Mar 23, 2022
  
  cad2c80a
- added timing for count-zeros & inner-step. · 80c91145
  Lawrence McAfee authored Mar 23, 2022
  
  80c91145
14 Mar, 2022 1 commit
- cleaned training.py · 787882a0
  Lawrence McAfee authored Mar 14, 2022
  
  787882a0
11 Mar, 2022 2 commits
- guard 'gather_params()' with 'if update_successful:' · f6811e28
  Lawrence McAfee authored Mar 11, 2022
  
  f6811e28
- tiny change. · c64098ef
  Lawrence McAfee authored Mar 10, 2022
  
  c64098ef
10 Mar, 2022 1 commit
- small fixes. · 862d70fc
  Lawrence McAfee authored Mar 10, 2022
  
  862d70fc
09 Mar, 2022 3 commits
- renamed reduce_grads/gather_params -> reduce_model_grads/gather_model_params · 64b9d28c
  Lawrence McAfee authored Mar 09, 2022
  
  64b9d28c
- passing args, timers from train_step to optimizer methods. · d58d1762
  Lawrence McAfee authored Mar 09, 2022
  
  d58d1762
- consolidated reduce_grad's sub-methods (i.e., allreduce_embedding_grads) · 45b364b1
  Lawrence McAfee authored Mar 09, 2022
  
  45b364b1
02 Mar, 2022 1 commit
- fixed gather params; now copying all params, not just local dp subset. · f8fd34e3
  Lawrence McAfee authored Mar 02, 2022
  
  f8fd34e3
01 Mar, 2022 1 commit
- fixed 'count zeros' for distrib opt · cbcd5579
  Lawrence McAfee authored Mar 01, 2022
  
  cbcd5579
23 Feb, 2022 1 commit
- isolated grad discrepency to BaseFloat16Optimizer.step() · 371a8828
  Lawrence McAfee authored Feb 23, 2022
  
  371a8828
22 Feb, 2022 1 commit
- debugging; localized issue to gather_params() · c13c0a3e
  Lawrence McAfee authored Feb 22, 2022
  
  c13c0a3e
15 Feb, 2022 4 commits
- adress review comments · 488f8c02
  Vijay Korthikanti authored Feb 11, 2022
  
  488f8c02
- minor fixes · 6f3bf9c0
  Vijay Korthikanti authored Feb 01, 2022
  
  6f3bf9c0
- vision third phase merge: pretraining methods + mit,swin backbones · 48c2a144
  Vijay Korthikanti authored Jan 31, 2022
  
  48c2a144
- wip; switching to grad-buffer-centric design · cb6f96b6
  Lawrence McAfee authored Feb 15, 2022
  
  cb6f96b6
11 Feb, 2022 1 commit
- adress review comments · 4554c3fe
  Vijay Korthikanti authored Feb 11, 2022
  
  4554c3fe
10 Feb, 2022 2 commits
- more work on Float16DistributedOptimizer · 49cca4d9
  Lawrence McAfee authored Feb 10, 2022
  
  49cca4d9
- working on Float16DistributedOptimizer · 329fe582
  Lawrence McAfee authored Feb 10, 2022
  
  329fe582
01 Feb, 2022 3 commits
- comments, cleanup. · b93bef00
  Lawrence McAfee authored Feb 01, 2022
  
  b93bef00
- minor fixes · e1f9c3a5
  Vijay Korthikanti authored Feb 01, 2022
  
  e1f9c3a5
- vision third phase merge: pretraining methods + mit,swin backbones · 01a82723
  Vijay Korthikanti authored Jan 31, 2022
  
  01a82723
31 Jan, 2022 1 commit
- working for t5 [ encoder embedding only ] · 3af6725d
  Lawrence McAfee authored Jan 31, 2022
  
  3af6725d
29 Jan, 2022 1 commit
- narrowed issue to pipeline rank 0, virtual pipeline rank >= 1 · 5bc9f889
  Lawrence McAfee authored Jan 28, 2022
  
  5bc9f889
28 Jan, 2022 2 commits
- more naming cleanup · 641408f5
  Vijay Korthikanti authored Jan 28, 2022
  
  641408f5
- changing class name AnnealingLR to OptimizerParamScheduler · 04ecc834
  Vijay Korthikanti authored Jan 28, 2022
  
  04ecc834
27 Jan, 2022 1 commit
- address review comments · 53931b8b
  Vijay Korthikanti authored Jan 27, 2022
  
  53931b8b
26 Jan, 2022 1 commit
- address review comments · 8acbbe25
  Vijay Korthikanti authored Jan 26, 2022
  
  8acbbe25
24 Jan, 2022 2 commits
- added args.transformer_pipeline_model_parallel_size · a06af061
  Lawrence McAfee authored Jan 24, 2022
  
  a06af061
- fixed args.virtual_pipeline_model_parallel_size · c2b7d0b3
  Lawrence McAfee authored Jan 24, 2022
  
  c2b7d0b3
22 Jan, 2022 1 commit
- second phase of vision code merge · d8c85650
  Vijay Korthikanti authored Jan 21, 2022
  
  d8c85650
19 Jan, 2022 1 commit
- flag for data parallel random initialization · a7ee77ea
  Vijay Korthikanti authored Jan 19, 2022
  
  a7ee77ea
12 Jan, 2022 1 commit
- Phase1 merge: vit optimizations + dataset enhancements + scaled_softmax kernel · 7a77abd9
  Vijay Korthikanti authored Jan 12, 2022
  
  7a77abd9
21 Dec, 2021 1 commit
- comments · 81ad628e
  Vijay Korthikanti authored Dec 21, 2021
  
  81ad628e
17 Dec, 2021 1 commit
- pipeline_fixes · 17843605
  Vijay Korthikanti authored Dec 17, 2021
  
  17843605
02 Dec, 2021 1 commit
- update training.py · e1b3fc8c
  zihanl authored Dec 01, 2021
  
  e1b3fc8c