"README_ORIGIN.md" did not exist on "6517e488d3b67d5154607fa88ab64af4f99c2224"
  • Masaki Kozuki's avatar
    Cherry-pick Megatron-LM's changes in pipeline model parallel for T5 (#1232) · 0e25fcc4
    Masaki Kozuki authored
    * update parallel_state
    
    * update pipeline common funcs - forward_step and backward_step
    
    * update pipelining w/o interleaving
    
    * type hint
    
    * merge utils into without_interleaving
    
    Motivation: functions in utils are only used by
    forward_backward_pipelining_without_interleaving
    
    * fix handling of `model_type`
    
    * fix import of DDP
    
    * update set_input_tensor method
    
    * fix
    
    * cosmetic
    
    * update model
    
    * refactor pipeline test scripts
    0e25fcc4
parallel_state.py 18.1 KB