-
Ma Zerun authored
* Add gradient cumulative optimizer fixes #190 * Update optimizer.py * Update optimizer.py * fix loss scale improperly in last equivalent_iter * Add `GradientCumulativeOptimizerHook` in `__init__.py`. * Add docstring of `GradientCumulativeOptimizerHook`. * Add type check, BN warning and resume warning. And fix typo, lint the code. * Add unit test * Update docstring example. * Change GradientCumulativeOptimizerHook `__init__` arguments. * Add GradientCumulativeOptimzierHook unit tests with IterBasedRunner. * Add GradientCumulativeFp16OptimizerHook. * Add unit tests of GradientCumulativeFp16OptimizerHook * Use '!=' instead of '>' to determine resume Co-authored-by:Zhiyuan Chen <this@zyc.ai>
357b0dfb