Skip to content

GitLab

Unverified Commit b4ac449a authored by Kebe's avatar Kebe Committed by GitHub
Browse files

[Misc] Merge the logs of pp layers partitions (#16225) 


Signed-off-by: default avatarKebe <mail@kebe7jun.com>
parent 8e5314a4
Show whitespace changes
Inline Side-by-side
Showing with 5 additions and 4 deletions
vllm/distributed/utils.py
View file @ b4ac449a
...@@ -102,10 +102,11 @@ def get_pp_indices(num_hidden_layers: int, pp_rank: int, ...@@ -102,10 +102,11 @@ def get_pp_indices(num_hidden_layers: int, pp_rank: int,
if remaining_layers := num_hidden_layers % pp_size: if remaining_layers := num_hidden_layers % pp_size:
for i in range(2, remaining_layers + 2): for i in range(2, remaining_layers + 2):
partitions[-i] += 1 partitions[-i] += 1
logger.info("Hidden layers were unevenly partitioned: %s", logger.info(
"Hidden layers were unevenly partitioned: [%s]. "
"This can be manually overridden using the "
"VLLM_PP_LAYER_PARTITION environment variable",
",".join(str(p) for p in partitions)) ",".join(str(p) for p in partitions))
logger.info("This can be manually overridden using the "
"VLLM_PP_LAYER_PARTITION environment variable")
start_layer = sum(partitions[:pp_rank]) start_layer = sum(partitions[:pp_rank])
end_layer = start_layer + partitions[pp_rank] end_layer = start_layer + partitions[pp_rank]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment