Skip to content

GitLab

Commit 6a0ef5b1 authored by mshoeybi's avatar mshoeybi
Browse files

pushed a fix for torch ddp

parent e8fb052f
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 0 deletions
megatron/arguments.py
View file @ 6a0ef5b1
...@@ -154,6 +154,10 @@ def parse_args(extra_args_provider=None, defaults={}, ...@@ -154,6 +154,10 @@ def parse_args(extra_args_provider=None, defaults={},
assert args.DDP_impl == 'local' assert args.DDP_impl == 'local'
assert args.use_contiguous_buffers_in_local_ddp assert args.use_contiguous_buffers_in_local_ddp
# For torch DDP, we do not use contiguous buffer
if args.DDP_impl == 'torch':
args.use_contiguous_buffers_in_local_ddp = False
if args.dataloader_type is None: if args.dataloader_type is None:
args.dataloader_type = 'single' args.dataloader_type = 'single'
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment