GitLab

* initial commit

* bug fix

* Move build_stats from common to keras main, because it is only applicable in keras

* remove tailing blank line

* add test for synth data

* add kwargs to init

* add kwargs to function invokation

* correctly pass kwargs

* debug

* debug

* debug

* fix super init

* bug fix

* fix local_flags

* fix import

* bug fix

* fix log_steps flag

* bug fix

* bug fix: add missing return value

* resolve double-defined flags

* lint fix

* move log_steps flag to benchmarK flag

* fix lint

* lint fix

* lint fix

* try flag core default values

* bug fix

* bug fix

* bug fix

* debug

* debug

* remove debug prints

* rename benchmark methods

* flag bug fix for synth benchmark

* Initial commit

* Fininshed

* bug fix

* bug fix

* bug fix

* Resolve review comments

* Typo fix

* resolve comments

* fix number error

* Resolve comments

* from NCF_input import NCFDataset for line 181

The type __NCFDataset__ is used in the type declaration on line 81 but it is never imported.

[flake8](http://flake8.pycqa.org) testing of https://github.com/tensorflow/models on Python 3.7.1

$ __flake8 . --count --select=E9,F63,F72,F82 --show-source --statistics__
```
./official/recommendation/data_preprocessing.py:180:3: F821 undefined name 'NCFDataset'
  # type: (str, str, dict, typing.Optional[str], bool, typing.Optional[str]) -> (NCFDataset, typing.Callable)
  ^
1    F821 undefined name 'NCFDataset'
1
```
__E901,E999,F821,F822,F823__ are the "_showstopper_" [flake8](http://flake8.pycqa.org) issues that can halt the runtime with a SyntaxError, NameError, etc. These 5 are different from most other flake8 issues which are merely "style violations" -- useful for readability but they do not effect runtime safety.
* F821: undefined name `name`
* F822: undefined name `name` in `__all__`
* F823: local variable name referenced before assignment
* E901: SyntaxError or IndentationError
* E999: SyntaxError -- failed to compile a file into an Abstract Syntax Tree

* int, int, data_pipeline.BaseDataConstructor

https://mypy.readthedocs.io/en/latest/cheat_sheet.html

[flake8](http://flake8.pycqa.org) testing of https://github.com/tensorflow/models on Python 3.7.1

$ __flake8 . --count --select=E9,F63,F72,F82 --show-source --statistics__
```
./official/recommendation/data_pipeline.py:346:41: F821 undefined name 'string'
               epoch_dir=None           # type: string
                                        ^
```

required by multi-node collective ops in eager mode.

tf.to_int32 raise deprecated warning.
change tf.to_int32(labels) to tf.cast(labels, tf.int32)

* Add `input_context` to `input_fn` in cifar10_main.

* Change sharding log message to be consistent with `dataset.shard` params.

* Lint

* Shard input for distribution strategy.

* Pass in input_context from real input_fn.

* Pass in input_context from real input_fn.

* Make pipeline id base 1 for better readability.

* Add support for TPUEstimator to data processing pipeline and add the ability to store epochs in user specified location.

* Fix ncf test for keras

* add a todo for batch_size and eval_batch_size for ncf keras

* lint fix

* fix typos

* Lint fix

* fix lint

* resolve pr comment

* resolve pr comment

* xla to bs=128 for num_gpu=8

* remove todo

* optimizer back to compat.v1

* add doc string to fix lint

* Adding LARS to ResNet

* Fixes for the LARS patch

* Fixes for the LARS patch

* more fixes

* 1 more fix

* s/CollectiveAllReduceStrategy/MultiWorkerMirroredStrategy

* More s/contrib.distribute/distribute.experimental

* Collective communication options in MultiWorkerMirroredStrategy.

* Minor fixes

* No checkpointing if multi worker.

* turn off checkpointing

* fix lint

* Mixed precision support

* Add TODOs

* fix resnet breakage and add keras end-to-end tests

* delint

* address PR comments

* tmp commit

* tmp commit

* first attempt (without eval)

* Bug fixes

* bug fixes

* training done

* Loss NAN, no eval

* Loss weight problem solved

* resolve the NAN loss problem

* Problem solved. Clean up needed

* Added a todo

* Remove debug prints

* Extract get_optimizer to ncf_common

* Move metrics computation back to neumf; use DS.scope api

* Extract DS.scope code to utils

* lint fixes

* Move obtaining DS above producer.start to avoid race condition

* move pt 1

* move pt 2

* Update the run script

* Wrap keras_model related code into functions

* Update the doc for softmax_logitfy and change the method name

* Resolve PR comments

* working version with: eager, DS, batch and no masks

* Remove git conflict indicator

* move reshape to neumf_model

* working version, not converge

* converged

* fix a test

* more lint fix

* more lint fix

* more lint fixes

* more lint fix

* Removed unused imports

* fix test

* dummy commit for kicking of checks

* fix lint issue

* dummy input to kick off checks

* dummy input to kick off checks

* add collective to dist strat

* addressed review comments

* add a doc string

* Added XLA test with a monkey-patched op to avoid OOM

* Added doc strings in Keras benchmarks to avoid Lint error

* s/CollectiveAllReduceStrategy/MultiWorkerMirroredStrategy

* More s/contrib.distribute/distribute.experimental

* Add benchmarks for thread tuning.

* Address comment/

* Add a comment.

Add root_data_dir to constructor of Resnet50KerasBenchmarkSynth and Resnet50KerasBenchmarkReal (#6259)

* Remove isintance change for contrib strategy

Replace it with class name check instead which should work regardless

* Add quotes for string

* fix quote type

This is needed to avoid breaking benchmark execution if PerfZero provides more
Named arguments before  the benchmark class constructor is updated.

* Allow user to specify root_data_dir in the benchmark class constructor

* Address comments

* Update official resnet for multi worker training with distribution strategies.

* Fixes for multi worker training.

* Fix call to `get_distribution_strategy`.

* Undo test change.

* Fix spacing.

* Move cluster configuration to distribution_utils.

* Move train_and_evaluate out of loop.  Also, update docstrings for multi-worker flags and add use_train_and_evaluate flag.

* Update distribution_strategy flag to match exported name for collective strategy.

* Add flag to enable XLA in Keras models

* Fix lint errors (some of them are old errors)