Skip to content

GitLab

Commit 39c80331 authored by Boris Fomitchev's avatar Boris Fomitchev
Browse files

fixed typo in comment 


Signed-off-by: default avatarBoris Fomitchev <bfomitchev@nvidia.com>
parent c632d205
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
megatron/initialize.py
View file @ 39c80331
...@@ -65,7 +65,7 @@ def initialize_megatron(extra_args_provider=None, args_defaults={}, ...@@ -65,7 +65,7 @@ def initialize_megatron(extra_args_provider=None, args_defaults={},
# delayed initialization of DDP-related stuff # delayed initialization of DDP-related stuff
# We only set basic DDP globals # We only set basic DDP globals
set_model_parallel_world_size(args.model_parallel_size) set_model_parallel_world_size(args.model_parallel_size)
# and refurn function for external DDP manager to call when it has DDP initialized # and return function for external DDP manager to call when it has DDP initialized
set_model_parallel_rank(args.rank) set_model_parallel_rank(args.rank)
return finish_mpu_init return finish_mpu_init
else: else:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment