"tests/wav2vec2_phoneme/test_tokenization_wav2vec2_phoneme.py" did not exist on "77d6c826d8365e631c804f013a76285facaaa954"
  • Deepak Narayanan's avatar
    Interleaved pipeline execution and code refactoring · dd889062
    Deepak Narayanan authored
    - Split a model's computation into multiple virtual stages as needed,
    and schedule communication correctly between these virtual stages
    - Move schedule code into `schedules.py` and communication code into
    `p2p_communication.py`
    - Use hyphens instead of spaces in all time logging for consistency
    - Factor out code in megatron/training.py into helper functions
    - Refactor evaluate() function: make it use forward_backward_schedule
    functions
    dd889062
pretrain_bert.py 6.18 KB