Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
ColossalAI
Commits
3263cdf5
Commit
3263cdf5
authored
Sep 08, 2022
by
Jiatong Han
Committed by
Frank Lee
Sep 08, 2022
Browse files
[NFC] polish colossalai/nn/parallel/data_parallel.py code style (#1570)
Co-authored-by:
JThh
<
jiatong.han@u.nus.edu
>
parent
f566c9b9
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
colossalai/nn/parallel/data_parallel.py
colossalai/nn/parallel/data_parallel.py
+2
-2
No files found.
colossalai/nn/parallel/data_parallel.py
View file @
3263cdf5
...
...
@@ -350,7 +350,7 @@ class ZeroDDP(ColoDDP):
for
tensor
in
chunk
.
get_tensors
():
rec_p
=
torch
.
empty
([
0
])
if
record_flag
:
rec_p
=
tensor
.
cpu
()
# move the whole tensor to CPU mem
rec_p
=
tensor
.
cpu
()
# move the whole tensor to CPU mem
assert
tensor
not
in
param_to_save_data
param_to_save_data
[
tensor
]
=
rec_p
# release the actual memory of the chunk
...
...
@@ -406,7 +406,7 @@ class ZeroDDP(ColoDDP):
state_dict
=
state_dict
.
copy
()
if
metadata
is
not
None
:
# mypy isn't aware that "_metadata" exists in state_dict
state_dict
.
_metadata
=
metadata
# type: ignore[attr-defined]
state_dict
.
_metadata
=
metadata
# type: ignore[attr-defined]
prefix
=
''
local_metadata
=
{}
if
metadata
is
None
else
metadata
.
get
(
prefix
[:
-
1
],
{})
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment