GitLab

Fix dispatch where we have a parameter group with multiple
combinations of types
Optionally apply weight decay after momentum

Incorrect types used in a few places

Only support the 4 specific cases we care about
Remove more general set of switch statements

Fuse in fp16 gradient -> fp32 convert
Additional option fp16 weight copy written out

Initial implementation, all fp32
Tested against torch.optim.sgd

Add "dynamic" argument to loss scaler

Unified mixed precision API + backend performance improvements

Bug fix in next power of 2

Fix minor typo