Unverified Commit 720d3363 authored by Igor Ganichev's avatar Igor Ganichev Committed by GitHub
Browse files

Fix order of variable update and moving average update

While the original code works fine in practice, it technically
allows gradient application and moving average update to happen
in any order. This causes the behavior to deviate from pure
mathematical specifications.
parents 2661eb97 f6e4dfe9
......@@ -370,12 +370,10 @@ def train(total_loss, global_step):
# Track the moving averages of all trainable variables.
variable_averages = tf.train.ExponentialMovingAverage(
MOVING_AVERAGE_DECAY, global_step)
variables_averages_op = variable_averages.apply(tf.trainable_variables())
with tf.control_dependencies([apply_gradient_op]):
variables_averages_op = variable_averages.apply(tf.trainable_variables())
with tf.control_dependencies([apply_gradient_op, variables_averages_op]):
train_op = tf.no_op(name='train')
return train_op
return variables_averages_op
def maybe_download_and_extract():
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment