Commit 96b7559d authored by Jared Casper's avatar Jared Casper
Browse files

Merge branch 'fix_backward_no_pipeline' into 'main'

fixed the timer issue for the case with no pipelining

See merge request ADLR/megatron-lm!456
parents b9ae7ba5 2514892c
...@@ -237,7 +237,7 @@ def forward_backward_no_pipelining(forward_step_func, ...@@ -237,7 +237,7 @@ def forward_backward_no_pipelining(forward_step_func,
timers, collect_non_loss_data) timers, collect_non_loss_data)
if not forward_only: if not forward_only:
backward_step(optimizer, input_tensor, output_tensor, backward_step(optimizer, input_tensor, output_tensor,
timers, output_tensor_grad) output_tensor_grad, timers)
# Run computation for last microbatch out of context handler (want to # Run computation for last microbatch out of context handler (want to
# synchronize gradients). # synchronize gradients).
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment