Skip to content

GitLab

Commit 5ae6008d authored by Michael Carilli's avatar Michael Carilli
Browse files

For testing purposes, enable the case where FusedAdam is not wrapped by amp

parent 53fd093d
Hide whitespace changes
Inline Side-by-side
Showing with 5 additions and 4 deletions
apex/optimizers/fused_adam.py
View file @ 5ae6008d
...@@ -78,10 +78,11 @@ class FusedAdam(torch.optim.Optimizer): ...@@ -78,10 +78,11 @@ class FusedAdam(torch.optim.Optimizer):
if closure is not None: if closure is not None:
loss = closure() loss = closure()
grads = self._amp_stash.grads if hasattr(self, "_amp_stash"):
output_params = self._amp_stash.output_params grads = self._amp_stash.grads
scale = self._amp_stash.scale output_params = self._amp_stash.output_params
grad_norms = self._amp_stash.grad_norms scale = self._amp_stash.scale
grad_norms = self._amp_stash.grad_norms
if grads is None: if grads is None:
grads_group = [None]*len(self.param_groups) grads_group = [None]*len(self.param_groups)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment