• Min Xu's avatar
    [feat] [FSDP]: add experimental support to shared weights (#836) · f2af4c66
    Min Xu authored
    
    
    * added a new test, passing without shared weights
    
    * tested weight sharing
    
    * added the test to test list file
    
    * extended to world_size = 2
    
    * fixed test
    
    * [feat]: add limited and experimental support for shared parameter
    
    * fixed tests
    
    * simplify to work with layer with at least 1 non-shared params and add code to pick up linked_param field for sharding the shared param
    
    * fixed the case where linked param is not in separate FSDP
    
    * changelog and remove old code
    Co-authored-by: default avatarMin Xu <min.xu.public@gmail.com>
    f2af4c66
parameter.pyi 768 Bytes