GitLab

* set strip_default_attrs=True for SavedModel exports

* specify dtype in resnet export

* another dtype fix

* fix another dtype issue, and set --image_bytes_as_serving_input to default to False

* Fix bug on distributed training in mnist using MirroredStrategy API

* Remove unnecessary codes and chagne distribution strategy source
- Remove multi-gpu
- Remove TowerOptimizer
- Change from MirroredStrategy to distribution_utils.get_distribution_strategy

* begin branch

* finish download script

* rename download to dataset

* intermediate commit

* intermediate commit

* misc tweaks

* intermediate commit

* intermediate commit

* intermediate commit

* delint and update census test.

* add movie tests

* delint

* fix py2 issue

* address PR comments

* intermediate commit

* intermediate commit

* intermediate commit

* finish wide deep transition to vanilla movielens

* delint

* intermediate commit

* intermediate commit

* intermediate commit

* intermediate commit

* fix import

* add default ncf csv construction

* change default on download_if_missing

* shard and vectorize example serialization

* fix import

* update ncf data unittests

* delint

* delint

* more delinting

* fix wide-deep movielens serialization

* address PR comments

* add file_io tests

* investigate wide-deep test failure

* remove hard coded path and properly use flags.

* address file_io test PR comments

* missed a hash_bucked_size

* Add DistributionStrategy to transformer model

* add num_gpu flag

* Calculate per device batch size for transformer

* remove reference to flags_core

* Add synthetic data option to transformer

* fix typo

* add import back in

* Use hierarchical copy

* address PR comments

* lint

* fix spaces

* group train op together to fix single GPU error

* Fix translate bug (sorted_keys is a dict, not a list)

* Change params to a default dict (translate.py was throwing errors because params didn't have the TPU parameters.)

* Address PR comments. Removed multi gpu flag + more

* fix lint

* fix more lints

* add todo for Synthetic dataset

* Update docs

* Revert 823da318. This restores distribution strategies for resnet.

This commit is not a direct revert due to significant merge conflict
resolution.

* fix flags test

* npc is no longer used in resnet

* fix treatment of main() and absl flags

* add mnist_eager, and delint

* add mnist_eager docstring

* squash of modular absl usage commits

* delint

* address PR comments

* change hooks to comma separated list, as absl behavior for space separated lists is not as expected

This reverts commit 32aa6563.

* begin transfer from contrib fork

more changes to resnet_run_loop

use AUTOTUNE in prefetch

first pass at resnet with functional distribution strategies

fix syntax error

delint

aesthetic tweaks

delint and fix typos

rip multi_gpu flag out of resnet entirely. Subject to saved model load verification

update cifar10 and imagenet tests to reflect that the model function no longer need to know about multi_gpu

fix imagenet test

start addressing PR comments

more PR response work

* misc tweaks

* add a comment

* final pr tweaks

* fix parsers

* Adding tests

* Adding tests

* Repackaging

* Adding logging

* Linting

* Updating name of logging package to avoid overwriting Python builtin logging.

* Updating name of logging package to avoid overwriting Python builtin logging.

* Adding export_dir and model saving for Resnet

* Moving to utils for tests

* Adding batch_size

* Adding multi-gpu export warning

* Responding to CR

* Py3 compliance

* move wide_deep parser

* move mnist parsers

* Glint everything

* Adding rcfile and pylinting

* Extra newline

* Few last lints

Use util functions hooks_helper and parser in mnist and wide_deep, and rename epochs_between_eval (from epochs_per_eval) (#3650)

* Adding logging utils

* restore utils

* delete old file

* update inputs and docstrings

* make /official a python module

* remove /utils directory

* Update readme for python path setting

* Change readme texts

Add an example showing how to train the MNIST model with eager execution
enabled.

(This change requires changes to TensorFlow made after
the 1.6 release branch was cut, i.e., will require a build
from source or TensorFlow 1.7+)

The `sparse` version is more efficient anyway.

I'm returning the labels shape [1] instead of []
because tf.accuracy fails otherwise.

Add multi-GPU option to MNIST

The `sparse` version is more efficient anyway.

I'm returning the labels shape [1] instead of []
because tf.accuracy fails otherwise.

This reverts commit e8726907.

Add multi-GPU flag to MNIST and allow for setting of replicated optimizer and model_fn

- Prior to this change, the use of tf.data.Dataset essentially embedded
  the entire training/evaluation dataset into the graph as a constant,
  leading to unnecessarily humungous graphs (Fixes #3017)
- Also, use batching on the evaluation dataset to allow
  evaluation on GPUs that cannot fit the entire evaluation dataset in
  memory (Fixes #3046)

This will make it easier to share the model definition with
eager execution and TPU demos without any side effects of running
unnecessary code on module import.

https://github.com/tensorflow/models/pull/3024

- Use the object-oriented tf.layers API instead of the functional one.
  The object-oriented API is particularly useful when using the model
  with eager execution.
- Update unittest to train, evaluate, and predict using the model.
- Add a micro-benchmark for measuring step-time.
  The parameters (batch_size, num_steps etc.) have NOT been tuned,
  the purpose with this code is mostly to illustrate how model
  benchmarks may be written.

These changes are made as a step towards consolidating model definitions
for different TensorFlow features (like eager execution and support for
TPUs in
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/eager/python/examples/mnist
and
https://github.com/tensorflow/tpu-demos/tree/master/cloud_tpu/models/mnist

With examples, and updates to the README

- Remove `convert_to_records.py` and instead create `tf.data.Dataset`
  objects directly from the numpy arrays.
- Format the Google Python Style (https://github.com/google/yapf/)