GitLab

FusedSGD now work as before
FusedAdam now work with o1/o2, no longer fuse scaling and casting
Removed special backend handling for FusedAdam
Moved and updated test for FusedAdam into run_optimizers
Removed legacy tests for optimizers.FP16_optimizer and FusedAdam in run_mixed_adam

fix fused layer norm for >65535 batch

Update FusedLayerNorm for new function API

fixing empty return from python implementation
  adding proper test to verify functional correctness for python implementation

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Fixing rank mapping for bn_group size == 8

fixing empty return from python implementation
  adding proper test to verify functional correctness for python implementation

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

https://github.com/NVIDIA/apex/issues/181#issuecomment-507958143

Add support for fp16 update term (new UPD_T typename in template)