GitLab

This reverts commit 63f5827d.

Add bisection based producer for increased scalability, enable fully deterministic data production, and use the materialized and bisection producer to check each other (via expected output md5's)

2nd half of rough replacement pass

fix dataset map functions

reduce bias in sample selection

cache pandas work on a daily basis

cleanup and fix batch check for multi gpu

multi device fix

fix treatment of eval data padding

print data producer

replace epoch overlap with padding and masking

move type and shape info into the producer class and update run.sh with larger batch size hyperparams

remove xla for multi GPU

more cleanup

remove model runner altogether

bug fixes

address subtle pipeline hang and improve producer __repr__

fix crash

fix assert

use popen_helper to create pools

add StreamingFilesDataset and abstract data storage to a separate class

bug fix

fix wait bug and add manual stack trace print

more bug fixes and refactor valid point mask to work with TPU sharding

misc bug fixes and adjust dtypes

address crash from decoding bools

fix remaining dtypes and change record writer pattern since it does not append

fix synthetic data

use TPUStrategy instead of TPUEstimator

minor tweaks around moving to TPUStrategy

cleanup some old code

delint and simplify permutation generation

remove low level tf layer definition, use single table with slice for keras, and misc fixes

missed minor point on removing tf layer definition

fix several bugs from recombinging layer definitions

delint and add docstrings

Update ncf_test.py. Section for identical inputs and different outputs was removed.

update data test to run against the new producer class

remove code that was migrated to tf/privacy

Fix variable name typo in OD g3 doc.

Fixes the variable name for the tf proto example function.

Remove broken links in detection model zoo.

Use the new distribution strategy API with keras (creating the model inside strategy.scope() instead of passing it to model.compile)

Fix benchmark imports and add license stmt

Cifar keras refactor

- kears_cifar_main returns dict of stats after run.
- unit tests added for keras_common.
- lint issues fixed in all 4 files.

 - Average of 5 runs single gpu is 93%+ range 92.5 to 93.4 top_1