Skip to content

GitLab

"docs/vscode:/vscode.git/clone" did not exist on "a103e3e9e2041add8bd83a8b5b35c497784b9722"
Commit 7feb02c6 authored by Jared Casper's avatar Jared Casper
Browse files

Merge branch 'apex-fix-fix' into 'master' 

Fixed APEX test

See merge request ADLR/megatron-lm!102
parents bf97f933 8f0af7cd
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
megatron/model/transformer.py
View file @ 7feb02c6
...@@ -21,7 +21,7 @@ import torch ...@@ -21,7 +21,7 @@ import torch
try: try:
from apex.normalization.fused_layer_norm import FusedLayerNorm as LayerNorm from apex.normalization.fused_layer_norm import FusedLayerNorm as LayerNorm
# Try to use FusedLayerNorm from Apex - this will trigger an error. # Try to use FusedLayerNorm from Apex - this will trigger an error.
_ = FusedLayerNorm(8, eps=1e-5) _ = LayerNorm(8, eps=1e-5)
except Exception as e: except Exception as e:
print('WARNING: APEX is not available, using torch.nn.LayerNorm instead of apex.normalization.FusedLayerNorm!') print('WARNING: APEX is not available, using torch.nn.LayerNorm instead of apex.normalization.FusedLayerNorm!')
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment