Skip to content

GitLab

Commit 4fcb2f45 authored by MaximumEntropy's avatar MaximumEntropy
Browse files

Fix spacing 


Signed-off-by: default avatarMaximumEntropy <sandeep.subramanian.1@umontreal.ca>
parent 1f8bc276
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
megatron/core/parallel_state.py
View file @ 4fcb2f45
...@@ -205,8 +205,8 @@ def initialize_model_parallel( ...@@ -205,8 +205,8 @@ def initialize_model_parallel(
if pipeline_model_parallel_split_rank is not None: if pipeline_model_parallel_split_rank is not None:
if ranks[pipeline_model_parallel_split_rank] not in embedding_ranks: if ranks[pipeline_model_parallel_split_rank] not in embedding_ranks:
embedding_ranks = [ranks[0], embedding_ranks = [ranks[0],
ranks[pipeline_model_parallel_split_rank], ranks[pipeline_model_parallel_split_rank],
ranks[-1]] ranks[-1]]
if ranks[pipeline_model_parallel_split_rank] not in position_embedding_ranks: if ranks[pipeline_model_parallel_split_rank] not in position_embedding_ranks:
position_embedding_ranks = [ranks[0], position_embedding_ranks = [ranks[0],
ranks[pipeline_model_parallel_split_rank]] ranks[pipeline_model_parallel_split_rank]]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment