GitLab

PiperOrigin-RevId: 264647492

Moving Keras ResNet models to `official/vision/image_classification` and benchmarks to `official/benchmark`.

PiperOrigin-RevId: 264268533

PiperOrigin-RevId: 264174913

PiperOrigin-RevId: 263837214

Also add `worker_hosts` and `task_index` flags.  These flags enable running the
model over multiple hosts by passing the cluster information via command line.

Setting `TF_CONFIG` will continue to work.

PiperOrigin-RevId: 263825245

* force_v2_in_keras_compile FLAG default to None and added seperate temp path.

* switch to force testing 1v path not force v2 path.

* Rename function force_v1_path.

* Force GPU placement for Resnet pure eager benchmark

* Create tweaked version of eager benchmark with explicit GPU placement

* Merged commit includes the following changes:
261380794  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261374439  by haoyuzhang<haoyuzhang@google.com>:

    Change Keras CTL dependencies from r1 ResNet to Keras ResNet.

--

PiperOrigin-RevId: 261380794

* Revert unintentional change

* Revert unintentional change

261339941  by haoyuzhang<haoyuzhang@google.com>:

    Own library functions in Keras ResNet models, and remove dependencies on v1 Estimator version of ResNet models.

    Most dependencies that the Keras version has are related to data input pipelines. Created dedicated files (cifar_preprocessing.py, imagenet_preprocessing.py) to collect all logic handling Cifar and ImageNet data input function.

--
261339166  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261317601  by akuegel<akuegel@google.com>:

    Internal change

261218818  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

PiperOrigin-RevId: 261339941

* Restructure resnet estimator code to under official/r1

* Continue moving resnet code...

* Improved README.md

261171038  by gjn<gjn@google.com>:

    Remove weight_decay_rate 0 early exit check

    Removing this code path should be fine since this was actually not doing
    what it meant to do. Since weight_decay_rate is actually a tensor, the
    equality check was only looking at the id of the object and comparing to
    0. This should never be true. Evaluating a tensor is also not what we
    want to do at this point of the code. Thus it should be fine to simply
    remove this code.

--
261169862  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261153520  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261140302  by hongkuny<hongkuny@google.com>:

    Clean up

--

PiperOrigin-RevId: 261171038

* Add accuracy and performance Resnet runs with --force_v2_in_keras_compile = True.

* Fixed lint

* Add force_run_distributed tests.

* Added enable_eager

* r/force_run_distributed/force_v2_in_keras_compile

* Adding force_v2 tests and FLAGs.

* Rename method to avoid conflict.

* Add cpu force_v2 tests.

* fix lint, wrap line.

* change to force_v2_in_keras_compile

* Update method name.

* Lower mlperf target to 0.736.

259030078  by isaprykin<isaprykin@google.com>:

    Clean up the --clone_model_in_keras_dist_strat from Keras Resnet.

    The cloning flag has been removed.  The current rule is that cloning is only done in graph mode.  That resulted in duplicate benchmarks: eager+no-cloning vs eager+cloning.  I removed eager+cloning ones.

--
259026454  by isaprykin<isaprykin@google.com>:

    Internal change

PiperOrigin-RevId: 259030078

* Merged commit includes the following changes:
258867180  by jingli<jingli@google.com>:

    Add new folders for upcoming reorg in model garden.

--
258893811  by hongkuny<hongkuny@google.com>:

    Adds summaries for metrics, allowing metrics inside keras.model.

--
258893048  by isaprykin<isaprykin@google.com>:

    Remove the `cloning` argument to `compile()`.

    Keras models are distributed by cloning in graph mode and without cloning in eager mode as of the change # 258652546.

--
258881002  by hongkuny<hongkuny@google.com>:

    Fix lint.

--
258874998  by hongkuny<hongkuny@google.com>:

    Internal

--
258872662  by hongkuny<hongkuny@google.com>:

    Fix doc

--

PiperOrigin-RevId: 258867180

* Create __init__.py

* Update __init__.py

* Update __init__.py

* Update __init__.py

* Config threadpool, cuDNN persistent BN, and grappler layout optimizer properly for ResNet56

* Add tweaked tests for Resnet56

* Avoid triggering the last partial batch overhead by explicitly dropping remainder

* Move to global_step.

* Hook to use global_step.

* fix comment start step 1 not step 0.

* remove hack used for testing.

* Add docstring.

* Improve performance for Cifar ResNet benchmarks

* Revert batch size changes to benchmarks

* Fix unit tests failures.

* 96% of TF 2.0 tests on GPU are passing.

* Currently all passing GPU and CPU TF 2.0

* Address code comments.

* use tf 2.0 cast.

* Comment about working on TF 2.0 CPU

* Uses contrib turn off for TF 2.0.

* Fix wide_deep and add keras_common_tests.

* use context to get num_gpus.

* Switch to tf.keras.metrics

* cpu benchmark and accuracy tests.

* add docstrings to fix lint.

* XLA FP32 and first test

* More XLA benchmarks FP32.

* Add eager to NCF and refactor resnet.

* fix v2_0 calls and more flag refactor.

* Remove extra flag args.

* 90 epoch default

* add return

* remove xla not used by estimator.

* Remove duplicate run_eagerly.

* fix flag defaults.

* Remove fp16_implementation flag option.

* Remove stop early on mlperf test.

* remove unneeded args.

* load flags from keras mains.

* Do not set learning phase when skipping eval

* Do not set learning phase in no dist strat case

* Added device placement, tweaked benchmarks

* Added tweaked benchmarks for Cifar

* Fix device scope

* Fix lint

* Add explicit GPU placement flag

* Also run accuracy test with explicit GPU placement

* Added doc string

* .

* .

* set default steps to 300K.

* Log flags to perfzero.

* Add XLA support to transformer

- Moved config logic to keras_utils
- Added enable_xla flag to _performance flags
- Did not refactor enable_xla flag from keras resnet due to
  reliance on calling FLAGs in estimator keras and that is
  a needed refactor for another time.

* fix g3 lint complaint.

* Refactor set config into keras_utils.

* Move flags out of main.

* pipe through enable_xla

* Update official/transformer/v2/misc.py
Co-Authored-By: Reed <reedwm@google.com>

* first version of ctl

* fix indent

* remove monkey patching for core

* add dtype arg

* fix dtype arg

* add logging lib

* remove compat.v1.logging

* add datetime import

* fix FLAGS import

* add constant vals

* move to using as tf import

* move to using as tf import

* remove steps per epoch = 1

* test train and test for one step

* test train and test for one step

* test train and test for one step

* test train and test for the entire dataset

* use an iterator for test

* pass tensors instead of an iterator

* add stats dict

* fix list declaration

* fix list declaration

* fix elapsed time calc

* print lr at epoch boundary alone

* Use regular tf import instead of compat

* remove tensorboard chkpts

* add correct logging import

* add correct logging import

* add benchmark configs

* add tests and configs

* add tests and configs

* add keras flags import

* add keras flags import

* fix eval ds creation cond

* return numpy value of train_loss

* return numpy value of loss and acc values

* add option for full eager mode

* fix lint errors

* add ctl flags

* add ctl import

* add the xla flag

* enable v2 behavior in unit tests

* rename dataset var

* add synthetic dataset without monkey patching

* add ctl local constants

* add ctl local constants

* change to using v2 imports

* change to using v2 imports

* change to using v2 imports

* change to using keras synthetic input fn

* remove enable_eager flag from benchmarks

* remove enable_eager flag from benchmarks

* remove enable_eager flag from benchmarks

* add option for no distrat

* add lambda for flags

* remove no_func benchmarks due to OOM error

* remove README

* remove unused comments

* remove unchanged file

* remove unchanged file

* remove unused drop_remainder_arg

* use keras.common lr function

* address PR comments

* remove reference to deleted file

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* fix lint errors

* .

* add flags info

* first version of ctl

* fix indent

* remove monkey patching for core

* add dtype arg

* fix dtype arg

* add logging lib

* remove compat.v1.logging

* add datetime import

* fix FLAGS import

* add constant vals

* move to using as tf import

* move to using as tf import

* remove steps per epoch = 1

* test train and test for one step

* test train and test for one step

* test train and test for one step

* test train and test for the entire dataset

* use an iterator for test

* pass tensors instead of an iterator

* add stats dict

* fix list declaration

* fix list declaration

* fix elapsed time calc

* print lr at epoch boundary alone

* Use regular tf import instead of compat

* remove tensorboard chkpts

* add correct logging import

* add correct logging import

* add benchmark configs

* add tests and configs

* add tests and configs

* add keras flags import

* add keras flags import

* fix eval ds creation cond

* return numpy value of train_loss

* return numpy value of loss and acc values

* add option for full eager mode

* fix lint errors

* add ctl flags

* add ctl import

* add the xla flag

* enable v2 behavior in unit tests

* rename dataset var

* add synthetic dataset without monkey patching

* add ctl local constants

* add ctl local constants

* change to using v2 imports

* change to using v2 imports

* change to using v2 imports

* change to using keras synthetic input fn

* remove enable_eager flag from benchmarks

* remove enable_eager flag from benchmarks

* remove enable_eager flag from benchmarks

* add option for no distrat

* add lambda for flags

* remove no_func benchmarks due to OOM error

* remove README

* remove unused comments

* remove unchanged file

* remove unchanged file

* remove unused drop_remainder_arg

* use keras.common lr function

* address PR comments

* remove reference to deleted file

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* fix lint errors

* .