Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "3f87f83116ccda4738b98db7a6dee4cfec3c78e4"
Switch branch/tag
Find file BlameHistoryPermalink
  • Deepak Narayanan's avatar
    Interleaved pipeline execution and code refactoring · dd889062
    Deepak Narayanan authored 
    - Split a model's computation into multiple virtual stages as needed,
and schedule communication correctly between these virtual stages
- Move schedule code into `schedules.py` and communication code into
`p2p_communication.py`
- Use hyphens instead of spaces in all time logging for consistency
- Factor out code in megatron/training.py into helper functions
- Refactor evaluate() function: make it use forward_backward_schedule
functions
    dd889062
pretrain_bert.py 6.18 KB