GitLab

* Add resnet56 short tests.
- created base benchmark module
- renamed accuracy test class to contain the word Accuracy
which will result in a need to update all the jobs
and a loss of history but is worth it.
- short tests are mostly copied from shining with oss refactor

* Address feedback.

* Move flag_methods to init
- Address setting default flags repeatedly.

* Rename accuracy tests.

* Lint errors resolved.

* fix model_dir set to flags.data_dir.

* fixed not fulling pulling out flag_methods.

Explicitly allow for script execution from any directory.  Make env vars visible in python script. (#6105)

* Update README.md

1. Clarify instructions to avoid friction. https://github.com/tensorflow/tensorflow/issues/24136
2. If trying to run in Colab experienced issue entering Python path. https://medium.com/@cwcgchello/how-to-set-pythonpath-in-colab-bf8c239c7c29

* Update README.md

* Change to gfile.Exists and deeper link to CIFAR-10

* fix lint and use tf.gfile

It currently fails with
TypeError: not all arguments converted during string formatting

* Added pass/fail and eager tests

* Add eager multi-gpu test.

* Single timestamp list and return average_exp_per_second.

* Add paren to subtract before division.

* Add timestamp history for each batch in training

* Resolve github comments

* Change the batch start recording logic

* Lint fix

* Add imagenet end-to-end accuracy test.

* Comment why ['foo']

* remove development test example.

* Add end-to-end tests for Estimator.

* comment from keras to estimator.

* remove trailing new line

* Set use_bias=False for conv layers in Keras ResNet model

* Removed bias regularizer from Conv2D layers (which should have no effect after bias is removed)

* Setting default data format based on available devices.

* Handle data format in Keras ResNet model properly for better performance on GPU; Compile only the training graph when skip_eval flag is True

* Added data format fix to Keras Cifar model; Removed unnecessary import

* Add a comment to the skip_eval flag per Priya's request

This reverts commit 63f5827d.

Add bisection based producer for increased scalability, enable fully deterministic data production, and use the materialized and bisection producer to check each other (via expected output md5's)

2nd half of rough replacement pass

fix dataset map functions

reduce bias in sample selection

cache pandas work on a daily basis

cleanup and fix batch check for multi gpu

multi device fix

fix treatment of eval data padding

print data producer

replace epoch overlap with padding and masking

move type and shape info into the producer class and update run.sh with larger batch size hyperparams

remove xla for multi GPU

more cleanup

remove model runner altogether

bug fixes

address subtle pipeline hang and improve producer __repr__

fix crash

fix assert

use popen_helper to create pools

add StreamingFilesDataset and abstract data storage to a separate class

bug fix

fix wait bug and add manual stack trace print

more bug fixes and refactor valid point mask to work with TPU sharding

misc bug fixes and adjust dtypes

address crash from decoding bools

fix remaining dtypes and change record writer pattern since it does not append

fix synthetic data

use TPUStrategy instead of TPUEstimator

minor tweaks around moving to TPUStrategy

cleanup some old code

delint and simplify permutation generation

remove low level tf layer definition, use single table with slice for keras, and misc fixes

missed minor point on removing tf layer definition

fix several bugs from recombinging layer definitions

delint and add docstrings

Update ncf_test.py. Section for identical inputs and different outputs was removed.

update data test to run against the new producer class