Unverified Commit 6df3e19b authored by ver217's avatar ver217 Committed by GitHub
Browse files

[hotfix] zero optim prevents calling inner optim.zero_grad (#1422)

parent 504419d2
......@@ -139,6 +139,7 @@ class ZeroOptimizer(ColossalaiOptimizer):
self._update_params_ptr()
ret = self.optim.step(*args, **kwargs)
self._register_states()
self.zero_grad()
self._update_fp16_params()
return ret
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment