GitLab

Removing patching of loss.backward, which appears to cause memory leaks (reference cycles?) in some models

apex.optimizers.FP16_Optimizer: add state_dict() and load_state_dict()

Restore fused kernel

allowing syncBN to run with affine = False

Update default dims in word_language_model to be multiples of 8 to enable Tensor Core use

Updated default sizes to be multiples of 8 to enable Tensor Core use.  Added performance guidelines to README.

add unit tests for optimizers/fp16_optimizer

don't convert to float bn with affine=False

[syncBN]

test update to resolve
  https://github.com/NVIDIA/apex/issues/134#issue-403525480

Using identical learning rate for both DDP with sync BN and single process BN.
The previous configure leaves the impression that sync BN requires adjusting lr
in the script, which is not true.

Fix + tests for the eval->training caching issue