• Matthew Yu's avatar
    support pytorch checkpoint as teacher model using config · dc176d58
    Matthew Yu authored
    Summary:
    Pull Request resolved: https://github.com/facebookresearch/d2go/pull/371
    
    In a previous iteration of this diff, we were specifying the teacher model in the same config as the student model, something like:
    ```
    # config.py
    MODEL:
      FBNET_V2:
      ...
    DISTILLATION:
      TEACHER:
        MODEL:
          FBNET_V2:
          ...
          WEIGHTS: /path/to/teacher/weights
    ...
    ```
    
    This leads to some oddities in the code, like we have to have a default config that adds all the required keys in the distillation teacher model.
    
    In this diff, we just let the user supply a teacher config (and optionally runner_name and overwrite opts) and use the supplied runner to build the model:
    ```
    # new_config.py
    MODEL:
      FBNET_V2:
    ...
    DISTILLATION:
      TEACHER:
        CONFIG_FNAME: /path/to/teacher/config
        RUNNER_NAME:
    ...
    ```
    
    This should make it very easy to specify the teacher as the user could potentially just reuse the trained_config generated in d2go.
    
    Reviewed By: newstzpz
    
    Differential Revision: D37640041
    
    fbshipit-source-id: 088a636c96f98279c9a04e32d1674f703451aec3
    dc176d58
test_modeling_distillation.py 13.8 KB