Update optimized_sync_batchnorm_kernel.py

in line 54, running_var should be running_variance..

Update optimized_sync_batchnorm_kernel.py
in line 54, running_var should be running_variance..
9b114c15 · Bingchen Zhao · GitHub · a8c2b7dd · 9b114c15
Unverified Commit 9b114c15 authored Mar 31, 2019 by Bingchen Zhao Committed by GitHub Mar 31, 2019
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

apex/parallel/optimized_sync_batchnorm_kernel.py apex/parallel/optimized_sync_batchnorm_kernel.py +1 -1

No files found.
--- a/apex/parallel/optimized_sync_batchnorm_kernel.py
+++ b/apex/parallel/optimized_sync_batchnorm_kernel.py
@@ -51,7 +51,7 @@ class SyncBatchnormFunction(Function):
            running_variance.data = running_variance.data * (1-momentum) + momentum*r_v_inc
        else:
            mean = running_mean.data
-            inv_std = 1.0 / torch.sqrt(running_var.data + eps)
+            inv_std = 1.0 / torch.sqrt(running_variance.data + eps)

        ctx.save_for_backward(input, weight, mean, inv_std)
        ctx.process_group = process_group