Skip to content

GitLab

Switch branch/tag
Find file BlameHistoryPermalink
  • Matthew Yu's avatar
    turn off interleaving if only saving on rank0 · 3111ae59
    Matthew Yu authored 
    Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/482

We should avoid using interleaving during save if we are calling save on one process:
```
if comm.is_main_process():
  save()
```
this is because interleave calls comm.synchronize() so will just wait indefinitely.

This diff updates the FSDP checkpointer to use save(interleave=False) when running on one process.

Reviewed By: wat3rBro, YanjunChen329

Differential Revision: D43526328

fbshipit-source-id: 672993a87af627aca090384b0c218798bd42fcde
    3111ae59
fsdp_checkpoint.py 9.37 KB