Document adam betas TrainingArguments (#8688)

63e91f5f · Sylvain Gugger · GitHub · 94caaa93 · 63e91f5f
Unverified Commit 63e91f5f authored Nov 20, 2020 by Sylvain Gugger Committed by GitHub Nov 20, 2020
Show whitespace changes
Inline Side-by-side

Showing with 4 additions and 0 deletions

src/transformers/training_args.py src/transformers/training_args.py +4 -0

No files found.
--- a/src/transformers/training_args.py
+++ b/src/transformers/training_args.py
@@ -89,6 +89,10 @@ class TrainingArguments:
            The initial learning rate for Adam.
        weight_decay (:obj:`float`, `optional`, defaults to 0):
            The weight decay to apply (if not zero).
+        adam_beta1 (:obj:`float`, `optional`, defaults to 0.9):
+            The beta1 for the Adam optimizer.
+        adam_beta2 (:obj:`float`, `optional`, defaults to 0.999):
+            The beta2 for the Adam optimizer.
        adam_epsilon (:obj:`float`, `optional`, defaults to 1e-8):
            Epsilon for the Adam optimizer.
        max_grad_norm (:obj:`float`, `optional`, defaults to 1.0):