Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Megatron-LM
Commits
0528bf7769f29a856a17234a15b806c044bc73d6
Switch branch/tag
megatron-lm
megatron
optimizer
distrib_optimizer.py
09 Mar, 2022
8 commits
more cleanup
· 0528bf77
Lawrence McAfee
authored
Mar 09, 2022
0528bf77
removed faulty (already commented out) gather_model_params param iterator.
· 94a90215
Lawrence McAfee
authored
Mar 09, 2022
94a90215
renamed reduce_grads/gather_params -> reduce_model_grads/gather_model_params
· 64b9d28c
Lawrence McAfee
authored
Mar 09, 2022
64b9d28c
passing args, timers from train_step to optimizer methods.
· d58d1762
Lawrence McAfee
authored
Mar 09, 2022
d58d1762
consolidated reduce_grad's sub-methods (i.e., allreduce_embedding_grads)
· 45b364b1
Lawrence McAfee
authored
Mar 09, 2022
45b364b1
passing 'model_parallel_group' to clip_grads, count_zeros
· 9546d8f0
Lawrence McAfee
authored
Mar 09, 2022
9546d8f0
encapsulated 'get_model_parallel_group()'; renamed BaseFloat16Optimizer -> MixedPrecisionOptimizer
· 772a4a2d
Lawrence McAfee
authored
Mar 09, 2022
772a4a2d
moved distrib opt to own file.
· 28b63ec7
Lawrence McAfee
authored
Mar 09, 2022
28b63ec7
