Skip to content

GitLab

Commit c55e154f authored by Mostofa Patwary's avatar Mostofa Patwary
Browse files

added adam betas and eps as arguments

parent 48269d8d
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
megatron/arguments.py
View file @ c55e154f
...@@ -206,7 +206,7 @@ def _add_regularization_args(parser): ...@@ -206,7 +206,7 @@ def _add_regularization_args(parser):
help='Second coefficient for computing running averages of' help='Second coefficient for computing running averages of'
'gradient and its square') 'gradient and its square')
group.add_argument('--adam-eps', type=float, default=1e-08, group.add_argument('--adam-eps', type=float, default=1e-08,
help='term added to the denominator to improve' help='Term added to the denominator to improve'
'numerical stability') 'numerical stability')
return parser return parser
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment