GitLab

No need to test explicit_gpu_placement + no_dist_strat + function case, since the performance issue is already fixed by unifying the Keras codepaths. However, there is still performance gap using the explicit_gpu_placement flag when run_eagerly.

PiperOrigin-RevId: 283890008

PiperOrigin-RevId: 282669615

This will help general debugging by enabling custom execution with  --benchmark_method_steps.

E.g --benchmark_method_steps=train_steps=7 will run the benchmark for only 7 steps without modifying benchmark code.

PiperOrigin-RevId: 282396875

PiperOrigin-RevId: 278902210

PiperOrigin-RevId: 277082247

PiperOrigin-RevId: 270122397

PiperOrigin-RevId: 270079001

PiperOrigin-RevId: 268836696

PiperOrigin-RevId: 268703439

PiperOrigin-RevId: 264511778

using existing flag --fp16_implementation in official/utils/flags/_performance.py to enable automatic mixed precision

Moving Keras ResNet models to `official/vision/image_classification` and benchmarks to `official/benchmark`.

PiperOrigin-RevId: 264268533

PiperOrigin-RevId: 264174913

Also add `worker_hosts` and `task_index` flags.  These flags enable running the
model over multiple hosts by passing the cluster information via command line.

Setting `TF_CONFIG` will continue to work.

PiperOrigin-RevId: 263825245

* force_v2_in_keras_compile FLAG default to None and added seperate temp path.

* switch to force testing 1v path not force v2 path.

* Rename function force_v1_path.

* Force GPU placement for Resnet pure eager benchmark

* Create tweaked version of eager benchmark with explicit GPU placement

* Add accuracy and performance Resnet runs with --force_v2_in_keras_compile = True.

* Fixed lint

* Add force_run_distributed tests.

* Added enable_eager

* r/force_run_distributed/force_v2_in_keras_compile

* Adding force_v2 tests and FLAGs.

* Rename method to avoid conflict.

* Add cpu force_v2 tests.

* fix lint, wrap line.

* change to force_v2_in_keras_compile

* Update method name.

* Lower mlperf target to 0.736.

259030078  by isaprykin<isaprykin@google.com>:

    Clean up the --clone_model_in_keras_dist_strat from Keras Resnet.

    The cloning flag has been removed.  The current rule is that cloning is only done in graph mode.  That resulted in duplicate benchmarks: eager+no-cloning vs eager+cloning.  I removed eager+cloning ones.

--
259026454  by isaprykin<isaprykin@google.com>:

    Internal change

PiperOrigin-RevId: 259030078

* Do not set learning phase when skipping eval

* Do not set learning phase in no dist strat case

* Added device placement, tweaked benchmarks

* Added tweaked benchmarks for Cifar

* Fix device scope

* Fix lint

* Add explicit GPU placement flag

* Also run accuracy test with explicit GPU placement

* Added doc string

* set default steps to 300K.

* Log flags to perfzero.

* Add XLA support to transformer

- Moved config logic to keras_utils
- Added enable_xla flag to _performance flags
- Did not refactor enable_xla flag from keras resnet due to
  reliance on calling FLAGs in estimator keras and that is
  a needed refactor for another time.

* fix g3 lint complaint.

* Refactor set config into keras_utils.

* Move flags out of main.

* pipe through enable_xla

* Update official/transformer/v2/misc.py
Co-Authored-By: Reed <reedwm@google.com>

* layout off for some tests and channels last.

* 8 gpu tests channels_last

* more layout off tests.

* Modify tweaked tests for better performance in no cloning mode

* Tweak trivial models

Because we run warmup tests in all real data benchmarks, XLA bugs will cause non-XLA tests to fail as well.

* Add mlperf like test.

* Final comments.

* docstring wording tweak.

* non-tweaked version

* Support pure eager execution in ResNet50

* Use smaller batch size

* Run different numbers of steps on different platforms

* Add new tests for delayed performance measurement

* Add a graph optional_next Reset benchmark.

* Fix lint error.

* Merged commit includes the following changes:
249776315  by tianlin<tianlin@google.com>:

    Internal change

249763206  by tianlin<tianlin@google.com>:

    For TF 2.0 (related to Beam Search), expand cond dims in tf.where(cond, x, y) to make all parameters broadcastable.

--
249392724  by hongkuny<hongkuny@google.com>:

    Internal change

PiperOrigin-RevId: 249776315

* Merged commit includes the following changes:
249823043  by tianlin<tianlin@google.com>:

    Bring back v2 test for predict and eval.

--

PiperOrigin-RevId: 249823043

* Add a test enabling get_next_as_optional behavior.

* Remove repeated flag.

* Remove trailing space.

* Make the name shorter.

* Fix lint error.

* Refine the benchmark name.