• Min Xu's avatar
    [refactor] enhance wrap and auto_wrap (#467) · a05a79bc
    Min Xu authored
    
    
    * [refactor] enhance wrap and auto_wrap
    
    - Two things were done in this PR
      1. We don't need to import FSDP in wrap.py since the wrapper class
         type is stored in the context now.
      2. We can use a `auto_wrap_policy` function to customize wrapping policy
         for auto_wrap, including size of module, blacklist, exclude list
    - The auto_wrap function got simplified a bit as a minor side effect.
    
    * Update fairscale/nn/wrap/auto_wrap.py
    Co-authored-by: default avatarSean Naren <sean@grid.ai>
    
    * addressed comments
    
    * addressed more comments
    Co-authored-by: default avatarSean Naren <sean@grid.ai>
    a05a79bc
test_wrap.py 7.01 KB