Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
b4ac449a
Unverified
Commit
b4ac449a
authored
Apr 08, 2025
by
Kebe
Committed by
GitHub
Apr 08, 2025
Browse files
[Misc] Merge the logs of pp layers partitions (#16225)
Signed-off-by:
Kebe
<
mail@kebe7jun.com
>
parent
8e5314a4
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
4 deletions
+5
-4
vllm/distributed/utils.py
vllm/distributed/utils.py
+5
-4
No files found.
vllm/distributed/utils.py
View file @
b4ac449a
...
...
@@ -102,10 +102,11 @@ def get_pp_indices(num_hidden_layers: int, pp_rank: int,
if
remaining_layers
:
=
num_hidden_layers
%
pp_size
:
for
i
in
range
(
2
,
remaining_layers
+
2
):
partitions
[
-
i
]
+=
1
logger
.
info
(
"Hidden layers were unevenly partitioned: %s"
,
","
.
join
(
str
(
p
)
for
p
in
partitions
))
logger
.
info
(
"This can be manually overridden using the "
"VLLM_PP_LAYER_PARTITION environment variable"
)
logger
.
info
(
"Hidden layers were unevenly partitioned: [%s]. "
"This can be manually overridden using the "
"VLLM_PP_LAYER_PARTITION environment variable"
,
","
.
join
(
str
(
p
)
for
p
in
partitions
))
start_layer
=
sum
(
partitions
[:
pp_rank
])
end_layer
=
start_layer
+
partitions
[
pp_rank
]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment