Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
ColossalAI
Commits
f83c4d65
Unverified
Commit
f83c4d65
authored
Jul 13, 2022
by
runluo
Committed by
GitHub
Jul 13, 2022
Browse files
[NFC] polish colossalai/nn/layer/wrapper/pipeline_wrapper.py code style (#1303)
parent
7696cead
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
4 deletions
+2
-4
colossalai/nn/layer/wrapper/pipeline_wrapper.py
colossalai/nn/layer/wrapper/pipeline_wrapper.py
+2
-4
No files found.
colossalai/nn/layer/wrapper/pipeline_wrapper.py
View file @
f83c4d65
...
...
@@ -6,6 +6,7 @@ from colossalai.core import global_context as gpc
class
PipelineSharedModuleWrapper
:
def
__init__
(
self
,
pipeline_ranks
:
Union
[
List
[
int
],
Tuple
[
int
]])
->
None
:
assert
len
(
pipeline_ranks
)
>
1
,
f
'Expect len(pipeline_ranks) > 1, got
{
len
(
pipeline_ranks
)
}
'
self
.
pipeline_ranks
=
pipeline_ranks
...
...
@@ -22,10 +23,7 @@ class PipelineSharedModuleWrapper:
num_pp_stages
=
num_dp_groups
//
pp_size
for
i
in
range
(
dp_size
):
for
j
in
range
(
num_pp_stages
):
pipeline_ranks
=
list
(
range
(
i
*
num_dp_groups
+
j
,
(
i
+
1
)
*
num_dp_groups
,
num_pp_stages
))
pipeline_ranks
=
list
(
range
(
i
*
num_dp_groups
+
j
,
(
i
+
1
)
*
num_dp_groups
,
num_pp_stages
))
sub_ranks
=
[
pipeline_ranks
[
idx
]
for
idx
in
self
.
pipeline_ranks
]
group
=
dist
.
new_group
(
sub_ranks
)
if
rank
in
sub_ranks
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment