GitLab

* Modify tweaked tests for better performance in no cloning mode

* Tweak trivial models

Because we run warmup tests in all real data benchmarks, XLA bugs will cause non-XLA tests to fail as well.

* Add mlperf like test.

* Final comments.

* docstring wording tweak.

* non-tweaked version

* Support pure eager execution in ResNet50

* Use smaller batch size

* Run different numbers of steps on different platforms

* Add new tests for delayed performance measurement

* Add a graph optional_next Reset benchmark.

* Fix lint error.

* Merged commit includes the following changes:
249776315  by tianlin<tianlin@google.com>:

    Internal change

249763206  by tianlin<tianlin@google.com>:

    For TF 2.0 (related to Beam Search), expand cond dims in tf.where(cond, x, y) to make all parameters broadcastable.

--
249392724  by hongkuny<hongkuny@google.com>:

    Internal change

PiperOrigin-RevId: 249776315

* Merged commit includes the following changes:
249823043  by tianlin<tianlin@google.com>:

    Bring back v2 test for predict and eval.

--

PiperOrigin-RevId: 249823043

* Add a test enabling get_next_as_optional behavior.

* Remove repeated flag.

* Remove trailing space.

* Make the name shorter.

* Fix lint error.

* Refine the benchmark name.

* Set the --clone_model_in_keras_dist_strat to None.  Remove the separate no_cloning benchmarks and add a couple of cloning ones.  Fixes the learning rate schedule to cache its ops per graph.

* Added 'tfdata_exp' version of all benchmarks which set
FLAGS.tf_data_experimental_slack = True. Renamed
`data_prefetch_with_slack` to `data_delay_prefetch` (haoyu's change)
to make the names more distinct.

* Add flag to resnet input pipeline and surface through
keras_imagenet_main.py

* Do not report metrics in performance benchmarks

* Rename flag

* Modified tweaked tests to use tensor learning rate

* Add benchmarks with the --cloning flag to Resnet and NFC.

* Renamed cloning to clone_model_in_keras_dist_strat. Dropped a few tests that aren't essential.

* Fixed up the formatting after re-naming the flag to a much longer  name.  Thanks, lint.
* Fixed the lint error in nfc_common.py

* Introduce a short sleep before ds.prefetch in tf.data.

* Further limit dataset threads to reduce CPU contention

* Tuned dataset sleep time

* Rename dataset sleep flag; enable it only for Keras Graph mode

* before moving test cases to the base class

* Added tests for keras cifar and keras imagenet

* fix cifar10_test

* add blank lines

* fix lint errors

* fix lint

* Resolve comments

* Modified two resnet keras tests

* Tests passed

* Remove keras_test_base

* Remove gpu from the no-dist tests

* add ds support for ncf

* remove comments for in_top_k

* avoid expanding the input layers

* resolve comments and fix lint

* Added some comments in code and fix lint

* fix lint

* add some documentation

* add tensorflow imports

* Added main function to the two keras resent benchmarks

* remove asserts

Forgot to add the `dtype='fp16'` flag in previous CL...

* Add benchmarks for dynamic loss scaling.

I add one convergence benchmark and five performance benchmarks.

* Fix lint error

* Add tweaked test for fp16 nonXLA

* Add dummy test for dataset warmup

* Add tests for Resnet data input pipeline

* Added trivial model class to isolate data input pipeline tests

* xla to bs=128 for num_gpu=8

* remove todo

* Added XLA test with a monkey-patched op to avoid OOM

* Added doc strings in Keras benchmarks to avoid Lint error

* Add benchmarks for thread tuning.

* Address comment/

* Add a comment.

Add root_data_dir to constructor of Resnet50KerasBenchmarkSynth and Resnet50KerasBenchmarkReal (#6259)