Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
apex
Commits
91362442
Commit
91362442
authored
Apr 25, 2019
by
Michael Carilli
Browse files
syntax
parent
75139ca3
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
apex/optimizers/fused_sgd.py
apex/optimizers/fused_sgd.py
+2
-2
No files found.
apex/optimizers/fused_sgd.py
View file @
91362442
...
@@ -126,7 +126,7 @@ class FusedSGD(Optimizer):
...
@@ -126,7 +126,7 @@ class FusedSGD(Optimizer):
first_runs
=
[
True
,
True
]
first_runs
=
[
True
,
True
]
# I think a bit of code divergence in exchange for naming clarity is worthwhile
# I think a bit of code divergence in exchange for naming clarity is worthwhile
if
explicit_master_params
if
explicit_master_params
:
stash
=
self
.
_amp_stash
stash
=
self
.
_amp_stash
fp16_model_params
=
[
p
for
p
in
stash
.
fp16_groups
[
gid
]
if
p
.
grad
is
not
None
]
fp16_model_params
=
[
p
for
p
in
stash
.
fp16_groups
[
gid
]
if
p
.
grad
is
not
None
]
...
@@ -136,7 +136,7 @@ class FusedSGD(Optimizer):
...
@@ -136,7 +136,7 @@ class FusedSGD(Optimizer):
fp32_from_fp16_momentums
,
first_runs
[
0
]
=
self
.
get_momentums
(
fp32_from_fp16_params
)
fp32_from_fp16_momentums
,
first_runs
[
0
]
=
self
.
get_momentums
(
fp32_from_fp16_params
)
fp32_params
=
[
p
for
p
in
stash
.
fp32_from_fp32_groups
[
gid
]
if
p
.
grad
is
not
None
]
fp32_params
=
[
p
for
p
in
stash
.
fp32_from_fp32_groups
[
gid
]
if
p
.
grad
is
not
None
]
fp32_grads
=
[
p
.
grad
for
p
in
stash
.
fp32_from_fp32_groups
[
gid
]
if
p
.
grad
is
not
None
)
]
fp32_grads
=
[
p
.
grad
for
p
in
stash
.
fp32_from_fp32_groups
[
gid
]
if
p
.
grad
is
not
None
]
fp32_momentums
,
first_runs
[
1
]
=
self
.
get_momentums
(
fp32_params
)
fp32_momentums
,
first_runs
[
1
]
=
self
.
get_momentums
(
fp32_params
)
launch_sets
=
[[
fp16_model_grads
,
fp32_from_fp16_params
,
fp32_from_fp16_momentums
,
fp16_model_params
],
launch_sets
=
[[
fp16_model_grads
,
fp32_from_fp16_params
,
fp32_from_fp16_momentums
,
fp16_model_params
],
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment