Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
apex
Commits
651150cb
Commit
651150cb
authored
Apr 18, 2019
by
Michael Carilli
Browse files
cleanup
parent
843cdbe0
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
3 additions
and
4 deletions
+3
-4
apex/optim/sgd.py
apex/optim/sgd.py
+2
-2
csrc/multi_tensor_sgd_kernel.cu
csrc/multi_tensor_sgd_kernel.cu
+1
-1
setup.py
setup.py
+0
-1
No files found.
apex/optim/sgd.py
View file @
651150cb
...
@@ -103,7 +103,7 @@ class SGD(Optimizer):
...
@@ -103,7 +103,7 @@ class SGD(Optimizer):
for
p
in
params
:
for
p
in
params
:
param_state
=
self
.
state
[
p
]
param_state
=
self
.
state
[
p
]
# torch.optim.SGD initializes momentum in the main loop, we have
# torch.optim.SGD initializes momentum in the main loop, we have
# to do it here, and track whether or not we've done so, so that
# to do it here, and track whether or not we've done so, so that
# momentum application can be skipped in the main kernel.
# momentum application can be skipped in the main kernel.
if
'momentum_buffer'
not
in
param_state
:
if
'momentum_buffer'
not
in
param_state
:
first_run
=
True
first_run
=
True
...
@@ -113,7 +113,7 @@ class SGD(Optimizer):
...
@@ -113,7 +113,7 @@ class SGD(Optimizer):
first_run
=
False
first_run
=
False
momentums
.
append
(
param_state
[
'momentum_buffer'
])
momentums
.
append
(
param_state
[
'momentum_buffer'
])
# We have all parameters now, split them into appropriate groups for
# We have all parameters now, split them into appropriate groups for
# parallel execution, following the 4 possible combos that the underlying
# parallel execution, following the 4 possible combos that the underlying
# kernels support:
# kernels support:
# grad_type, param_type, momentum_type, requires_fp16_copy
# grad_type, param_type, momentum_type, requires_fp16_copy
...
...
csrc/multi_tensor_sgd_kernel.cu
View file @
651150cb
...
@@ -49,7 +49,7 @@ struct SGDFunctor
...
@@ -49,7 +49,7 @@ struct SGDFunctor
T_grad
*
grad_in
=
(
T_grad
*
)
tl
.
addresses
[
0
][
tensor_loc
];
T_grad
*
grad_in
=
(
T_grad
*
)
tl
.
addresses
[
0
][
tensor_loc
];
grad_in
+=
chunk_idx
*
chunk_size
;
grad_in
+=
chunk_idx
*
chunk_size
;
T_weight
*
weight_in
=
(
T_weight
*
)
tl
.
addresses
[
1
][
tensor_loc
];
T_weight
*
weight_in
=
(
T_weight
*
)
tl
.
addresses
[
1
][
tensor_loc
];
weight_in
+=
chunk_idx
*
chunk_size
;
weight_in
+=
chunk_idx
*
chunk_size
;
...
...
setup.py
View file @
651150cb
...
@@ -98,7 +98,6 @@ if "--cuda_ext" in sys.argv:
...
@@ -98,7 +98,6 @@ if "--cuda_ext" in sys.argv:
'nvcc'
:[
'-maxrregcount=50'
,
'nvcc'
:[
'-maxrregcount=50'
,
'-O3'
,
'-O3'
,
'--use_fast_math'
]
+
version_ge_1_1
}))
'--use_fast_math'
]
+
version_ge_1_1
}))
print
(
ext_modules
)
setup
(
setup
(
name
=
'apex'
,
name
=
'apex'
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment