Skip to content

GitLab

"docs/task_table.md" did not exist on "db1d67f612bc6bddd013480d90ccd7cde53c73f7"
Commit 1b28a51b authored by Lawrence McAfee's avatar Lawrence McAfee
Browse files

removed assert that prevented virtual pipelinining to be used with distributed checkpointing

parent be0da070
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 3 deletions
megatron/arguments.py
View file @ 1b28a51b
...@@ -264,9 +264,6 @@ def parse_args(extra_args_provider=None, defaults={}, ...@@ -264,9 +264,6 @@ def parse_args(extra_args_provider=None, defaults={},
assert args.activations_checkpoint_method is not None, \ assert args.activations_checkpoint_method is not None, \
'for distributed checkpoint activations to work you '\ 'for distributed checkpoint activations to work you '\
'need to use a activation-checkpoint method ' 'need to use a activation-checkpoint method '
assert args.num_layers_per_virtual_pipeline_stage is None, \
'currently distributed checkpoint activations only supported for ' \
'nointerleaved pipeline parallelism'
assert TORCH_MAJOR >= 1 and TORCH_MINOR >= 10, \ assert TORCH_MAJOR >= 1 and TORCH_MINOR >= 10, \
'distributed checkpoint activations are supported for pytorch ' \ 'distributed checkpoint activations are supported for pytorch ' \
'v1.10 and above (Nvidia Pytorch container >= 21.07). Current ' \ 'v1.10 and above (Nvidia Pytorch container >= 21.07). Current ' \
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment