Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Megatron-LM
Commits
d58d176284f284affa7b3f03eea386ddd5da43ed
Switch branch/tag
megatron-lm
megatron
optimizer
distrib_optimizer.py
09 Mar, 2022
5 commits
passing args, timers from train_step to optimizer methods.
· d58d1762
Lawrence McAfee
authored
Mar 09, 2022
d58d1762
consolidated reduce_grad's sub-methods (i.e., allreduce_embedding_grads)
· 45b364b1
Lawrence McAfee
authored
Mar 09, 2022
45b364b1
passing 'model_parallel_group' to clip_grads, count_zeros
· 9546d8f0
Lawrence McAfee
authored
Mar 09, 2022
9546d8f0
encapsulated 'get_model_parallel_group()'; renamed BaseFloat16Optimizer -> MixedPrecisionOptimizer
· 772a4a2d
Lawrence McAfee
authored
Mar 09, 2022
772a4a2d
moved distrib opt to own file.
· 28b63ec7
Lawrence McAfee
authored
Mar 09, 2022
28b63ec7
