Skip to content

GitLab

"git@developer.sourcefind.cn:orangecat/ollama.git" did not exist on "74788b487c3c3aa48715dfeb700871c61a63836d"
Commit 945ece94 authored by Jared Casper's avatar Jared Casper
Browse files

Merge branch 't5_pipeline_fixes' into 'main' 

t5_pipeline_fix

See merge request ADLR/megatron-lm!353
parents d4169684 ea128da5
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
megatron/mpu/initialize.py
View file @ 945ece94
...@@ -177,7 +177,7 @@ def initialize_model_parallel(tensor_model_parallel_size_=1, ...@@ -177,7 +177,7 @@ def initialize_model_parallel(tensor_model_parallel_size_=1,
if len(ranks) > 1: if len(ranks) > 1:
embedding_ranks = [ranks[0], ranks[-1]] embedding_ranks = [ranks[0], ranks[-1]]
if pipeline_model_parallel_split_rank_ is not None and \ if pipeline_model_parallel_split_rank_ is not None and \
pipeline_model_parallel_split_rank_ not in embedding_ranks: ranks[pipeline_model_parallel_split_rank_] not in embedding_ranks:
embedding_ranks = [ranks[0], embedding_ranks = [ranks[0],
ranks[pipeline_model_parallel_split_rank_], ranks[pipeline_model_parallel_split_rank_],
ranks[-1]] ranks[-1]]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment