GitLab

All models which support loss scaling support dynamic loss scaling, so the argument has no purpose. It used to be that some models scaled the loss manually instead of using a LossScaleOptimizer, and so did not support dynamic loss scaling.

PiperOrigin-RevId: 367719521

PiperOrigin-RevId: 359994674

PiperOrigin-RevId: 359990341

PiperOrigin-RevId: 326286926

PiperOrigin-RevId: 326286926

--stop_threshold, --num_gpu, --hooks, --export_dir, and --distribution_strategy have been unexposed from models which do not use them

PiperOrigin-RevId: 268032080

--clean, --train_epochs, and --epochs_between_evals have been unexposed from models which do not use them

PiperOrigin-RevId: 267065651

--synthetic_data, --dtype, --all_reduce_alg, and --num_packs have been unexposed from models which do not use them

PiperOrigin-RevId: 265483564

--num_parallel_calls, --inter_op_parallelism_threads, and --intra_op_parallelism_threads have been unexposed from models which do not use them

PiperOrigin-RevId: 264965788

Before, there was a global default loss scale for all models. Currently, only resnet uses loss scaling, but this will be useful once more models support it.

This will allow one to easily reproduce a benchmark by running with the flags.

* Add FP16 and benchmarks.

* add missing run and report.

* Add loss_scale as option not included with dtype.

* move loss_scale validation under dtype conditional.

* add loss_scale to flags tested.

* Add DistributionStrategy to transformer model

* add num_gpu flag

* Calculate per device batch size for transformer

* remove reference to flags_core

* Add synthetic data option to transformer

* fix typo

* add import back in

* Use hierarchical copy

* address PR comments

* lint

* fix spaces

* group train op together to fix single GPU error

* Fix translate bug (sorted_keys is a dict, not a list)

* Change params to a default dict (translate.py was throwing errors because params didn't have the TPU parameters.)

* Address PR comments. Removed multi gpu flag + more

* fix lint

* fix more lints

* add todo for Synthetic dataset

* Update docs

* Revert 823da318. This restores distribution strategies for resnet.

This commit is not a direct revert due to significant merge conflict
resolution.

* fix flags test

* npc is no longer used in resnet

* squash of modular absl usage commits

* delint

* address PR comments

* change hooks to comma separated list, as absl behavior for space separated lists is not as expected

This reverts commit 32aa6563.

* begin transfer from contrib fork

more changes to resnet_run_loop

use AUTOTUNE in prefetch

first pass at resnet with functional distribution strategies

fix syntax error

delint

aesthetic tweaks

delint and fix typos

rip multi_gpu flag out of resnet entirely. Subject to saved model load verification

update cifar10 and imagenet tests to reflect that the model function no longer need to know about multi_gpu

fix imagenet test

start addressing PR comments

more PR response work

* misc tweaks

* add a comment

* final pr tweaks

* fix parsers

* Add fp16 support to resnet.

* address PR comments

* add dtype checking to model definition

* delint

* more PR comments

* few more tweaks

* update resnet checkpoints

* Add benchmark upload util to bigquery.

Also update the benchmark logger and bigquery schema for the
errors found during the integration test.

* Fix lint error.

* Update test to clear all the env vars during test.

This was causing error since the Kokoro test has TF_PKG=tf-nightly
injected during test.

* Update lintrc to ignore google related package.

* Another attempt to fix lint import error.

* Address the review comment.

* Fix lint error.

* Another fix for lint.

* Update test comment for env var clean up.

* Update reset model for benchmark logging.

To enable benchmark logging, just add "--hooks LoggingMetricHook"

* Benchmark logger fix for resnet.

1. Update default at_end to False for metric logger to avoid
checkpoint error.
2. Update resnet run to log final evaluation result.

* Update log output for final eval_result.

* Typo fix.

* Unset the default value for benchmark_log_dir.

Usually the benchmark should be logged to different directly for
each run. Having a default value will hide the choice from user.

* Bug fix for benchmark logger initialization.

* Fix lint error.

* Address the review comment.

1. Update the logger to cover evaluation result.
2. Move the flag to performance parser.

* Undo the change for arg_parser.

Use util functions hooks_helper and parser in mnist and wide_deep, and rename epochs_between_eval (from epochs_per_eval) (#3650)

* Update resnet with logging utils

* intermediate commit

* commit before rebase from master

* Add tests of ExamplePerSecondHook

* Done with test

* Fix a style nit

* Fix a style nit

Create groups of arg parsers and convert the official resnet model to
the new arg parsers.