Commit f6e4dfe9 authored by Igor Ganichev's avatar Igor Ganichev
Browse files

Fix order of variable update and moving average update

While the original code works fine in practice, it technically
allows gradient application and moving average update to happen
in any order. This causes the behavior to deviate from pure
mathematical specifications.
parent 2661eb97
...@@ -370,12 +370,10 @@ def train(total_loss, global_step): ...@@ -370,12 +370,10 @@ def train(total_loss, global_step):
# Track the moving averages of all trainable variables. # Track the moving averages of all trainable variables.
variable_averages = tf.train.ExponentialMovingAverage( variable_averages = tf.train.ExponentialMovingAverage(
MOVING_AVERAGE_DECAY, global_step) MOVING_AVERAGE_DECAY, global_step)
variables_averages_op = variable_averages.apply(tf.trainable_variables()) with tf.control_dependencies([apply_gradient_op]):
variables_averages_op = variable_averages.apply(tf.trainable_variables())
with tf.control_dependencies([apply_gradient_op, variables_averages_op]): return variables_averages_op
train_op = tf.no_op(name='train')
return train_op
def maybe_download_and_extract(): def maybe_download_and_extract():
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment