GitLab

* Restructure resnet estimator code to under official/r1

* Continue moving resnet code...

* Improved README.md

261171038  by gjn<gjn@google.com>:

    Remove weight_decay_rate 0 early exit check

    Removing this code path should be fine since this was actually not doing
    what it meant to do. Since weight_decay_rate is actually a tensor, the
    equality check was only looking at the id of the object and comparing to
    0. This should never be true. Evaluating a tensor is also not what we
    want to do at this point of the code. Thus it should be fine to simply
    remove this code.

--
261169862  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261153520  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261140302  by hongkuny<hongkuny@google.com>:

    Clean up

--

PiperOrigin-RevId: 261171038

* Add accuracy and performance Resnet runs with --force_v2_in_keras_compile = True.

* Fixed lint

* Add force_run_distributed tests.

* Added enable_eager

* r/force_run_distributed/force_v2_in_keras_compile

* Adding force_v2 tests and FLAGs.

* Rename method to avoid conflict.

* Add cpu force_v2 tests.

* fix lint, wrap line.

* change to force_v2_in_keras_compile

* Update method name.

* Lower mlperf target to 0.736.

259030078  by isaprykin<isaprykin@google.com>:

    Clean up the --clone_model_in_keras_dist_strat from Keras Resnet.

    The cloning flag has been removed.  The current rule is that cloning is only done in graph mode.  That resulted in duplicate benchmarks: eager+no-cloning vs eager+cloning.  I removed eager+cloning ones.

--
259026454  by isaprykin<isaprykin@google.com>:

    Internal change

PiperOrigin-RevId: 259030078

* Merged commit includes the following changes:
258867180  by jingli<jingli@google.com>:

    Add new folders for upcoming reorg in model garden.

--
258893811  by hongkuny<hongkuny@google.com>:

    Adds summaries for metrics, allowing metrics inside keras.model.

--
258893048  by isaprykin<isaprykin@google.com>:

    Remove the `cloning` argument to `compile()`.

    Keras models are distributed by cloning in graph mode and without cloning in eager mode as of the change # 258652546.

--
258881002  by hongkuny<hongkuny@google.com>:

    Fix lint.

--
258874998  by hongkuny<hongkuny@google.com>:

    Internal

--
258872662  by hongkuny<hongkuny@google.com>:

    Fix doc

--

PiperOrigin-RevId: 258867180

* Create __init__.py

* Update __init__.py

* Update __init__.py

* Update __init__.py

* Config threadpool, cuDNN persistent BN, and grappler layout optimizer properly for ResNet56

* Add tweaked tests for Resnet56

* Avoid triggering the last partial batch overhead by explicitly dropping remainder

* Move to global_step.

* Hook to use global_step.

* fix comment start step 1 not step 0.

* remove hack used for testing.

* Add docstring.

* Improve performance for Cifar ResNet benchmarks

* Revert batch size changes to benchmarks

* Fix unit tests failures.

* 96% of TF 2.0 tests on GPU are passing.

* Currently all passing GPU and CPU TF 2.0

* Address code comments.

* use tf 2.0 cast.

* Comment about working on TF 2.0 CPU

* Uses contrib turn off for TF 2.0.

* Fix wide_deep and add keras_common_tests.

* use context to get num_gpus.

* Switch to tf.keras.metrics

* cpu benchmark and accuracy tests.

* add docstrings to fix lint.

* XLA FP32 and first test

* More XLA benchmarks FP32.

* Add eager to NCF and refactor resnet.

* fix v2_0 calls and more flag refactor.

* Remove extra flag args.

* 90 epoch default

* add return

* remove xla not used by estimator.

* Remove duplicate run_eagerly.

* fix flag defaults.

* Remove fp16_implementation flag option.

* Remove stop early on mlperf test.

* remove unneeded args.

* load flags from keras mains.

* Do not set learning phase when skipping eval

* Do not set learning phase in no dist strat case

* Added device placement, tweaked benchmarks

* Added tweaked benchmarks for Cifar

* Fix device scope

* Fix lint

* Add explicit GPU placement flag

* Also run accuracy test with explicit GPU placement

* Added doc string

* .

* .

* set default steps to 300K.

* Log flags to perfzero.

* Add XLA support to transformer

- Moved config logic to keras_utils
- Added enable_xla flag to _performance flags
- Did not refactor enable_xla flag from keras resnet due to
  reliance on calling FLAGs in estimator keras and that is
  a needed refactor for another time.

* fix g3 lint complaint.

* Refactor set config into keras_utils.

* Move flags out of main.

* pipe through enable_xla

* Update official/transformer/v2/misc.py
Co-Authored-By: Reed <reedwm@google.com>

* first version of ctl

* fix indent

* remove monkey patching for core

* add dtype arg

* fix dtype arg

* add logging lib

* remove compat.v1.logging

* add datetime import

* fix FLAGS import

* add constant vals

* move to using as tf import

* move to using as tf import

* remove steps per epoch = 1

* test train and test for one step

* test train and test for one step

* test train and test for one step

* test train and test for the entire dataset

* use an iterator for test

* pass tensors instead of an iterator

* add stats dict

* fix list declaration

* fix list declaration

* fix elapsed time calc

* print lr at epoch boundary alone

* Use regular tf import instead of compat

* remove tensorboard chkpts

* add correct logging import

* add correct logging import

* add benchmark configs

* add tests and configs

* add tests and configs

* add keras flags import

* add keras flags import

* fix eval ds creation cond

* return numpy value of train_loss

* return numpy value of loss and acc values

* add option for full eager mode

* fix lint errors

* add ctl flags

* add ctl import

* add the xla flag

* enable v2 behavior in unit tests

* rename dataset var

* add synthetic dataset without monkey patching

* add ctl local constants

* add ctl local constants

* change to using v2 imports

* change to using v2 imports

* change to using v2 imports

* change to using keras synthetic input fn

* remove enable_eager flag from benchmarks

* remove enable_eager flag from benchmarks

* remove enable_eager flag from benchmarks

* add option for no distrat

* add lambda for flags

* remove no_func benchmarks due to OOM error

* remove README

* remove unused comments

* remove unchanged file

* remove unchanged file

* remove unused drop_remainder_arg

* use keras.common lr function

* address PR comments

* remove reference to deleted file

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* fix lint errors

* .

* add flags info

* first version of ctl

* fix indent

* remove monkey patching for core

* add dtype arg

* fix dtype arg

* add logging lib

* remove compat.v1.logging

* add datetime import

* fix FLAGS import

* add constant vals

* move to using as tf import

* move to using as tf import

* remove steps per epoch = 1

* test train and test for one step

* test train and test for one step

* test train and test for one step

* test train and test for the entire dataset

* use an iterator for test

* pass tensors instead of an iterator

* add stats dict

* fix list declaration

* fix list declaration

* fix elapsed time calc

* print lr at epoch boundary alone

* Use regular tf import instead of compat

* remove tensorboard chkpts

* add correct logging import

* add correct logging import

* add benchmark configs

* add tests and configs

* add tests and configs

* add keras flags import

* add keras flags import

* fix eval ds creation cond

* return numpy value of train_loss

* return numpy value of loss and acc values

* add option for full eager mode

* fix lint errors

* add ctl flags

* add ctl import

* add the xla flag

* enable v2 behavior in unit tests

* rename dataset var

* add synthetic dataset without monkey patching

* add ctl local constants

* add ctl local constants

* change to using v2 imports

* change to using v2 imports

* change to using v2 imports

* change to using keras synthetic input fn

* remove enable_eager flag from benchmarks

* remove enable_eager flag from benchmarks

* remove enable_eager flag from benchmarks

* add option for no distrat

* add lambda for flags

* remove no_func benchmarks due to OOM error

* remove README

* remove unused comments

* remove unchanged file

* remove unchanged file

* remove unused drop_remainder_arg

* use keras.common lr function

* address PR comments

* remove reference to deleted file

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* fix lint errors

* .

* tf.compat.v1.train.experimental.enable_mixed_precision_graph_rewrite

* Remove num_parallel_batches which is not used.

* layout off for some tests and channels last.

* 8 gpu tests channels_last

* more layout off tests.

* Add 1 gpu force_eager benchmark

* Add accuracy for no dist strat eager

* remvove return.

Before, there was a global default loss scale for all models. Currently, only resnet uses loss scaling, but this will be useful once more models support it.

* Modify tweaked tests for better performance in no cloning mode

* Tweak trivial models

* Add multi-worker benchmarks to official resnet estimator_benchmark.py.

* fix super constructor calls

* set datasets_num_private_threads to 32 in multi worker tweaked benchmarks

Because we run warmup tests in all real data benchmarks, XLA bugs will cause non-XLA tests to fail as well.

* Add mlperf like test.

* Final comments.

* docstring wording tweak.

* non-tweaked version