apex/contrib/optimizers/distributed_fused_lamb.py · 8a80d478c8f64fcaec071c3fc2ff50cd17f22576 · OpenDAS / apex

Distributed LAMB fixes (#1007) · 8a80d478

Kexin Yu authored Dec 03, 2020



* add flag for DistributedAdam: step_support_amp_scaling
Co-authored-by: Kexin Yu <kexiny@nvidia.com>
Co-authored-by: Kexin Yu <kexinznzn@gmail.com>

8a80d478

distributed_fused_lamb.py 35.3 KB

Replace distributed_fused_lamb.py