Skip to content

GitLab

Unverified Commit f29efb77 authored by Sudhakar Singh's avatar Sudhakar Singh Committed by GitHub
Browse files

fix for amax_and_scale_update when reduce_amax=False (#386) 


Signed-off-by: default avatarSudhakar Singh <sudhakars@nvidia.com>
parent 7444946d
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
transformer_engine/pytorch/module/base.py
View file @ f29efb77
...@@ -78,7 +78,7 @@ def _prepare_backward( ...@@ -78,7 +78,7 @@ def _prepare_backward(
# Update amax and scale; Skip all setup for global amax reduction # Update amax and scale; Skip all setup for global amax reduction
if not fp8_meta["recipe"].reduce_amax: if not fp8_meta["recipe"].reduce_amax:
FP8GlobalStateManager.amax_and_scale_update(fp8_meta, False) amax_and_scale_update(fp8_meta, False)
else: else:
# From previous iteration # From previous iteration
FP8GlobalStateManager.copy_amax_from_global_buffer(fp8_meta, forward=False) FP8GlobalStateManager.copy_amax_from_global_buffer(fp8_meta, forward=False)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment