• Min Xu's avatar
    [test] Add unittest for checkpoint & DDP (#126) · 6658be22
    Min Xu authored
    * Add unittest for checkpoint & DDP
    
    - this change adds test cases to reproduce the error with checkpoint & DDP
    - mandeep mentioned that there is also deadlock in this case, but this
      change doesn't cover that.
    - we cover cases where weight sharing is OK
    - however, same module multiple checkpoint or find_unused_parameters are
      both not OK
    
    * added norm checks
    6658be22
test_checkpoint_ddp.py 5.86 KB