Skip to content

GitLab

You need to sign in or sign up before continuing.
Commit 1328bc09 authored by Alexander Rives's avatar Alexander Rives Committed by Facebook Github Bot
Browse files

add missing condition for first moment type as statement 

Summary: Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/684

Differential Revision: D16006333

Pulled By: myleott

fbshipit-source-id: 95bd4215734281194008fa029e81407d63b335ac
parent 8b514b9f
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 1 deletion
fairseq/optim/adafactor.py
View file @ 1328bc09
...@@ -165,7 +165,8 @@ class Adafactor(torch.optim.Optimizer): ...@@ -165,7 +165,8 @@ class Adafactor(torch.optim.Optimizer):
state['RMS'] = 0 state['RMS'] = 0
else: else:
state['exp_avg'] = state['exp_avg'].type_as(grad) if use_first_moment:
state['exp_avg'] = state['exp_avg'].type_as(grad)
if factored: if factored:
state['exp_avg_sq_row'] = state['exp_avg_sq_row'].type_as(grad) state['exp_avg_sq_row'] = state['exp_avg_sq_row'].type_as(grad)
state['exp_avg_sq_col'] = state['exp_avg_sq_col'].type_as(grad) state['exp_avg_sq_col'] = state['exp_avg_sq_col'].type_as(grad)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment