Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Megatron-LM
Commits
e055f3cf
Commit
e055f3cf
authored
Feb 22, 2021
by
Vijay Korthikanti
Browse files
fix warning condition
parent
a1f4032b
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
megatron/arguments.py
megatron/arguments.py
+1
-1
No files found.
megatron/arguments.py
View file @
e055f3cf
...
@@ -223,7 +223,7 @@ def parse_args(extra_args_provider=None, defaults={},
...
@@ -223,7 +223,7 @@ def parse_args(extra_args_provider=None, defaults={},
custom_kernel_constraint
=
seq_len
>
16
and
seq_len
<=
2048
and
\
custom_kernel_constraint
=
seq_len
>
16
and
seq_len
<=
2048
and
\
seq_len
%
4
==
0
and
attn_batch_size
%
4
==
0
seq_len
%
4
==
0
and
attn_batch_size
%
4
==
0
if
args
.
fp16
and
custom_kernel_constraint
and
args
.
masked_softmax_fusion
:
if
not
(
args
.
fp16
and
custom_kernel_constraint
and
args
.
masked_softmax_fusion
)
:
print
(
'WARNING: constraints for invoking optimized'
print
(
'WARNING: constraints for invoking optimized'
' fused softmax kernel are not met. We default back to unfused'
' fused softmax kernel are not met. We default back to unfused'
' kernel invocations.'
)
' kernel invocations.'
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment