Commits · 7ac342b704d4cf6d5391bb6f6ef32cba51cc8972 · OpenDAS / Megatron-LM · GitLab

09 Mar, 2022 9 commits
- removed old (pre-gbuf-padding) get_model_grad_buffer_dp_views(). · 7ac342b7
  Lawrence McAfee authored Mar 09, 2022
  
  7ac342b7
- more cleanup · 0528bf77
  Lawrence McAfee authored Mar 09, 2022
  
  0528bf77
- removed faulty (already commented out) gather_model_params param iterator. · 94a90215
  Lawrence McAfee authored Mar 09, 2022
  
  94a90215
- renamed reduce_grads/gather_params -> reduce_model_grads/gather_model_params · 64b9d28c
  Lawrence McAfee authored Mar 09, 2022
  
  64b9d28c
- passing args, timers from train_step to optimizer methods. · d58d1762
  Lawrence McAfee authored Mar 09, 2022
  
  d58d1762
- consolidated reduce_grad's sub-methods (i.e., allreduce_embedding_grads) · 45b364b1
  Lawrence McAfee authored Mar 09, 2022
  
  45b364b1
- passing 'model_parallel_group' to clip_grads, count_zeros · 9546d8f0
  Lawrence McAfee authored Mar 09, 2022
  
  9546d8f0
- encapsulated 'get_model_parallel_group()'; renamed BaseFloat16Optimizer -> MixedPrecisionOptimizer · 772a4a2d
  Lawrence McAfee authored Mar 09, 2022
  
  772a4a2d
- moved distrib opt to own file. · 28b63ec7
  Lawrence McAfee authored Mar 09, 2022
  
  28b63ec7