Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
apex
Commits
b6188fc4
Commit
b6188fc4
authored
Dec 11, 2018
by
Deyu Fu
Browse files
address comment
parent
d665ab90
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
4 deletions
+1
-4
apex/optimizers/fused_adam.py
apex/optimizers/fused_adam.py
+1
-4
No files found.
apex/optimizers/fused_adam.py
View file @
b6188fc4
...
@@ -100,10 +100,7 @@ class FusedAdam(torch.optim.Optimizer):
...
@@ -100,10 +100,7 @@ class FusedAdam(torch.optim.Optimizer):
if
clip
>
1
:
if
clip
>
1
:
combined_scale
=
clip
*
scale
combined_scale
=
clip
*
scale
# set bias correction for this group
bias_correction
=
1
if
group
[
'bias_correction'
]
else
0
bias_correction
=
0
if
group
[
'bias_correction'
]:
bias_correction
=
1
for
p
,
grad
,
output_param
in
zip
(
group
[
'params'
],
grads_this_group
,
output_params_this_group
):
for
p
,
grad
,
output_param
in
zip
(
group
[
'params'
],
grads_this_group
,
output_params_this_group
):
#note: p.grad should not ever be set for correct operation of mixed precision optimizer that sometimes sends None gradients
#note: p.grad should not ever be set for correct operation of mixed precision optimizer that sometimes sends None gradients
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment