colossalai/zero/low_level/low_level_optim.py · c0a033700c7027286ecd8a7bcbaafc6f794323ad · OpenDAS / ColossalAI

[shardformer] fix master param sync for hybrid plugin/rewrite unwrapping logic (#4758) · c0a03370

Baizhou Zhang authored Sep 20, 2023

* fix master param sync for hybrid plugin

* rewrite unwrap for ddp/fsdp

* rewrite unwrap for zero/gemini

* rewrite unwrap for hybrid plugin

* fix geemini unwrap

* fix bugs

c0a03370

low_level_optim.py 26.5 KB

Replace low_level_optim.py