check disable_observer before updating observers

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/407

As is, observers can be periodically updated even if observer is disabled. Add a check on whether observer is disabled before updating observers periodically.

Reviewed By: tglik

Differential Revision: D40734564

fbshipit-source-id: fd3e50e95c95ad39bd993f71e02dc7acc79744cc

d2go/quantization/modeling.py

@@ -613,6 +613,7 @@ class QATHook(HookBase):
 
         if (
             self._applied["enable_fake_quant"]
+            and not self._applied["disable_observer"]
             and cfg.QUANTIZATION.QAT.UPDATE_OBSERVER_STATS_PERIODICALLY
             and cur_iter % cfg.QUANTIZATION.QAT.UPDATE_OBSERVER_STATS_PERIOD == 0
         ):