• FoolPlayer's avatar
    [shardformer] init shardformer code structure (#3731) · 6a69b44d
    FoolPlayer authored
    * init shardformer code structure
    
    * add implement of sharder (inject and replace)
    
    * add implement of replace layer to colossal layer
    
    * separate different layer policy, add some notion
    
    * implement 1d and 2d slicer, can tell col or row
    
    * fix bug when slicing and inject model
    
    * fix some bug; add inference test example
    6a69b44d
utils.py 1.48 KB