Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
fairscale
Commits
b33e705e
Unverified
Commit
b33e705e
authored
May 12, 2022
by
Min Xu
Committed by
GitHub
May 12, 2022
Browse files
[minor] fixing exception on the error path (#984)
Co-authored-by:
Min Xu
<
min.xu.public@gmail.com
>
parent
ae8a0bca
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
1 deletion
+5
-1
fairscale/optim/adascale.py
fairscale/optim/adascale.py
+5
-1
No files found.
fairscale/optim/adascale.py
View file @
b33e705e
...
@@ -147,6 +147,10 @@ class AdaScale(Optimizer):
...
@@ -147,6 +147,10 @@ class AdaScale(Optimizer):
num_gradients_to_accumulate
:
int
=
1
,
num_gradients_to_accumulate
:
int
=
1
,
debias_ewma
:
bool
=
True
,
debias_ewma
:
bool
=
True
,
):
):
# Init hook_handles list, otherwise, a partial init'ed object may fail in ``__del__``.
self
.
_hook_handles
:
List
[
Any
]
=
[]
# Init other fields.
self
.
_optimizer
=
optimizer
self
.
_optimizer
=
optimizer
self
.
_local_grad_sqr
:
Optional
[
torch
.
Tensor
]
=
None
self
.
_local_grad_sqr
:
Optional
[
torch
.
Tensor
]
=
None
self
.
_world_size
:
int
=
(
self
.
_world_size
:
int
=
(
...
@@ -183,7 +187,7 @@ class AdaScale(Optimizer):
...
@@ -183,7 +187,7 @@ class AdaScale(Optimizer):
self
.
_scale
=
1.0
# Assign to inform mypy about the typing of this variable.
self
.
_scale
=
1.0
# Assign to inform mypy about the typing of this variable.
self
.
set_scale
(
self
.
_world_size
*
self
.
_num_grads_to_accum
if
scale
is
None
else
scale
)
self
.
set_scale
(
self
.
_world_size
*
self
.
_num_grads_to_accum
if
scale
is
None
else
scale
)
self
.
_hook_handles
:
List
[
Any
]
=
[]
# Safer to register hooks after all init actions are done.
self
.
_hook
()
self
.
_hook
()
def
_hook
(
self
)
->
None
:
def
_hook
(
self
)
->
None
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment