GitLab

Add multi-GPU option to MNIST

* Using nightly TF docker

* Explicitly pull image

* Update readme

* Respond to CR

This reverts commit e8726907.

Add multi-GPU flag to MNIST and allow for setting of replicated optimizer and model_fn

Opening gzipped datasets in binary, read-only mode fixes the issue

This is a step towards merging the example in
https://github.com/tensorflow/tpu-demos/tree/master/cloud_tpu/models/mnist
with this repository, so we have a single model definition for
training across CPU/GPU/eager execution/TPU.

The change to dataset.py is so that the raw data can be read
from cloud storage systems (like GCS and S3).

- Prior to this change, the use of tf.data.Dataset essentially embedded
  the entire training/evaluation dataset into the graph as a constant,
  leading to unnecessarily humungous graphs (Fixes #3017)
- Also, use batching on the evaluation dataset to allow
  evaluation on GPUs that cannot fit the entire evaluation dataset in
  memory (Fixes #3046)

This will make it easier to share the model definition with
eager execution and TPU demos without any side effects of running
unnecessary code on module import.

https://github.com/tensorflow/models/pull/3024

- Use the object-oriented tf.layers API instead of the functional one.
  The object-oriented API is particularly useful when using the model
  with eager execution.
- Update unittest to train, evaluate, and predict using the model.
- Add a micro-benchmark for measuring step-time.
  The parameters (batch_size, num_steps etc.) have NOT been tuned,
  the purpose with this code is mostly to illustrate how model
  benchmarks may be written.

These changes are made as a step towards consolidating model definitions
for different TensorFlow features (like eager execution and support for
TPUs in
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/eager/python/examples/mnist
and
https://github.com/tensorflow/tpu-demos/tree/master/cloud_tpu/models/mnist

With examples, and updates to the README

- Remove `convert_to_records.py` and instead create `tf.data.Dataset`
  objects directly from the numpy arrays.
- Format the Google Python Style (https://github.com/google/yapf/)