Use FSDP.STATE_DICT_TYPE = SHARDED_STATE_DICT by default

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/535

Use `FSDP.STATE_DICT_TYPE = SHARDED_STATE_DICT` for FSDP checkpointing by default.` FSDP.USE_LOCAL_STATE_DICT` will be deprecated in the future.

# Note
After the change, config usage of `FSDP.USE_LOCAL_STATE_DICT` will not be picked up by code: it will be superseded by the default type of FSDP.STATE_DICT_TYPE instead

Reviewed By: tglik

Differential Revision: D45413143

fbshipit-source-id: e7bc2d5dc04ac09004cb89353333be020a9c80b5

configs/faster_rcnn_fbnetv3a_C4_FSDP.yaml

@@ -11,7 +11,7 @@ DATALOADER:
   NUM_WORKERS: 0
 FSDP:
   ALGORITHM: "full"
-  USE_LOCAL_STATE_DICT: True
+  STATE_DICT_TYPE: "SHARDED_STATE_DICT"
   # AUTO_WRAP_POLICY: ""
   STATE_DICT_CPU_OFFLOAD: False
   STATE_DICT_RANK0_ONLY: True

d2go/trainer/fsdp.py

@@ -55,10 +55,10 @@ def add_fsdp_configs(_C: CN):
     # A list of layer cls names to wrap, case sensitive
     _C.FSDP.AUTO_WRAP_LAYER_CLS = []
     # Whether to use local state dict -- superseded by STATE_DICT_TYPE
-    _C.FSDP.USE_LOCAL_STATE_DICT = False
+    _C.FSDP.USE_LOCAL_STATE_DICT = True
     # State dict type to use when calling FSDPWrapper.state_dict() (used when saving).
     # If None, defaults to checking the value of USE_LOCAL_STATE_DICT
-    _C.FSDP.STATE_DICT_TYPE = None
+    _C.FSDP.STATE_DICT_TYPE = "SHARDED_STATE_DICT"
     # Whether to offload state dict to cpu
     _C.FSDP.STATE_DICT_CPU_OFFLOAD = False
     # Whether to materialize state dict on rank 0