• ver217's avatar
    [tensor] ColoTensor supports ZeRo (#1015) · 9492a561
    ver217 authored
    * impl chunk manager
    
    * impl param op hook
    
    * add reduce_chunk
    
    * add zero hook v2
    
    * add zero dp
    
    * fix TensorInfo
    
    * impl load balancing when using zero without chunk
    
    * fix zero hook
    
    * polish chunk
    
    * fix bugs
    
    * ddp ok
    
    * zero ok
    
    * polish code
    
    * fix bugs about load balancing
    
    * polish code
    
    * polish code
    
    * add ene-to-end test
    
    * polish code
    
    * polish code
    
    * polish code
    
    * fix typo
    
    * add test_chunk
    
    * fix bugs
    
    * fix bugs
    
    * polish code
    9492a561
colo_parameter.py 3.5 KB