Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
ColossalAI
Commits
f0aa191f
Unverified
Commit
f0aa191f
authored
Feb 13, 2023
by
ver217
Committed by
GitHub
Feb 13, 2023
Browse files
[gemini] fix colo_init_context (#2683)
parent
5cd8cae0
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
colossalai/utils/model/colo_init_context.py
colossalai/utils/model/colo_init_context.py
+2
-2
No files found.
colossalai/utils/model/colo_init_context.py
View file @
f0aa191f
...
@@ -32,7 +32,7 @@ def _convert_to_coloparam(param: torch.nn.Parameter,
...
@@ -32,7 +32,7 @@ def _convert_to_coloparam(param: torch.nn.Parameter,
default_pg
:
Optional
[
ProcessGroup
]
=
None
,
default_pg
:
Optional
[
ProcessGroup
]
=
None
,
default_dist_spec
:
Optional
[
Any
]
=
None
)
->
ColoParameter
:
default_dist_spec
:
Optional
[
Any
]
=
None
)
->
ColoParameter
:
if
isinstanc
e
(
param
,
ColoParameter
)
:
if
typ
e
(
param
)
is
ColoParameter
:
return
param
return
param
# detaching tensor is necessary for optimizers.
# detaching tensor is necessary for optimizers.
requires_grad
=
param
.
requires_grad
requires_grad
=
param
.
requires_grad
...
@@ -102,7 +102,7 @@ class ColoInitContext(InsertPostInitMethodToModuleSubClasses):
...
@@ -102,7 +102,7 @@ class ColoInitContext(InsertPostInitMethodToModuleSubClasses):
"""
"""
name_list
=
[]
name_list
=
[]
for
name
,
param
in
_named_params_with_replica
(
module
):
for
name
,
param
in
_named_params_with_replica
(
module
):
if
isinstanc
e
(
param
,
Colo
Tensor
)
:
if
typ
e
(
param
)
is
Colo
Parameter
:
continue
continue
split
=
name
.
rfind
(
'.'
)
split
=
name
.
rfind
(
'.'
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment