Commit a4b513b1 authored by Hongkun Yu's avatar Hongkun Yu Committed by A. Unique TensorFlower
Browse files

Typo reported by https://github.com/tensorflow/models/pull/9672

PiperOrigin-RevId: 362836016
parent 0a17932d
...@@ -113,7 +113,7 @@ class AdamWeightDecay(tf.keras.optimizers.Adam): ...@@ -113,7 +113,7 @@ class AdamWeightDecay(tf.keras.optimizers.Adam):
correct way of using L2 regularization/weight decay with Adam, since that will correct way of using L2 regularization/weight decay with Adam, since that will
interact with the m and v parameters in strange ways. interact with the m and v parameters in strange ways.
Instead we want ot decay the weights in a manner that doesn't interact with Instead we want to decay the weights in a manner that doesn't interact with
the m/v parameters. This is equivalent to adding the square of the weights to the m/v parameters. This is equivalent to adding the square of the weights to
the loss with plain (non-momentum) SGD. the loss with plain (non-momentum) SGD.
""" """
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment