GitLab

250779087  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Reduce BERT Perfzero benchmark test training steps.

--

PiperOrigin-RevId: 250779087

* Support pure eager execution in ResNet50

* Use smaller batch size

250713045  by hongkuny<hongkuny@google.com>:

    TPU util

--

PiperOrigin-RevId: 250713045

250606180  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Fix BERT benchamrk test errors.

--
250589623  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Change BERT benchmark test pretrained checkpoint url.

--
250587892  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Fix error in BERT custom training loop checkpoint restoration.

--
250577163  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Add logic to inject callback that measures performance in BERT custom training
    loop.

--
250529526  by hongkuny<hongkuny@google.com>:

    Internal clean up

--
250428976  by hongkuny<hongkuny@google.com>:

    Internal change

250415383  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Add min/max value to BERT classifier benchmark test.

--
250376246  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Add benchmark performance test to run BERT on multiple numbers of GPUs.

--

PiperOrigin-RevId: 250606180

* Put all python dependencies into one line.

This makes it easier to copy, paste & install all dependencies at once. In addition many users have custom setups (virtualenv, conda, .etc). Having it in one line easily allows to grap the dependencies.

* Remove 'sudo' from all pip install commands and adjust troubleshooting section.

* Make max_length and static_batch configurable.

* Fix line length.

* Fix incorrect parameters in building eval input.

* Improve comments for readability.

* Add flag to use custom training loop for keras NCF model.

* Add error check to NCF model for custom training loop + tf1.0.

* Add static batch benchmarks to estimator 

So we can distinguish how much static vs dynamic batch matter.

* change max_length for static_batch tests

* Add flag for max length

* Add a custom training loop for NCF model with TF2.0

* Fix long line in ncf_keras_main.py

* Remove dataset repeat when using custom training loop.

this is not going to help with current tf.data semantics. so removing it.

* Fixes that make transformer run.

* Remove debug print statements.

* Changed the permissions to 644.

* Fix the rest of the permissions.

* enable static batch in all benchmarks

* Restrict dist strat hack to training mode

For now we will do predict/eval without dist strat, so remove that hack in non training cases.

* Use `inputs` instead of `x` as arg name for call

Keras has different behavior based on whether the inputs are called `inputs` or not. Using `inputs` gives expected behaviors.

* Avoid extra map fn on input in dist strat case

* Update how we handle custom metrics

This new approach works with and without dist strat. The previous one didn't work with dist strat. We need to fix that but this is reasonable in meantime (b/133724664).

* Update benchmarks

* typo in metrics code

* Revert metrics change

Didn't actually work in distributed case..

250347237  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Fix linting errors in BERT benchmark test.

--
250326131  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

250315593  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

250303528  by haoyuzhang<haoyuzhang@google.com>:

    Add method docstring to fix lint error.

--

PiperOrigin-RevId: 250347237

* Run different numbers of steps on different platforms

* Add new tests for delayed performance measurement

This shuffling should help in getting shuffling each epoch.

The ".mat" files loaded in the dataset are byte files. Python 3.7 requires them to be loaded using "rb".

250009207  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Add feature in BERT to write training metrics to a summary file.

--

PiperOrigin-RevId: 250009207

249896208  by hongkuny<hongkuny@google.com>:

    Adds __init__.py

--

PiperOrigin-RevId: 249896208

Add early stopping logic to ncf keras when desired threshold is met. Also change the default batch size to match the tuned hyperparams

249883771  by hongkuny<hongkuny@google.com>:

    Creates a benchmark dir

--

PiperOrigin-RevId: 249883771

* Moved common keras code to utils.

* Initial 1 gpu benchmark

- Aligned flags with resnet example
- removed code/features that are not super useful
- eval as part of train if bleu source/ref provided
- add exp_per_second hook

* Rename benchmark classes, pass batch-size and log_steps.

* fix docstring

* Predict done with checkpoints inline

- perfzero baseclass

* steps not epochs with smoother training loop.

* do not initialize history outside loop.

* 5000 between eval not 500

* estimator to keras.

* remove epochs var.

* use range not xrange.

* 200K steps for 1 gpu

* fix global step

* Add a graph optional_next Reset benchmark.

* Fix lint error.

* Merged commit includes the following changes:
249776315  by tianlin<tianlin@google.com>:

    Internal change

249763206  by tianlin<tianlin@google.com>:

    For TF 2.0 (related to Beam Search), expand cond dims in tf.where(cond, x, y) to make all parameters broadcastable.

--
249392724  by hongkuny<hongkuny@google.com>:

    Internal change

PiperOrigin-RevId: 249776315

* Merged commit includes the following changes:
249823043  by tianlin<tianlin@google.com>:

    Bring back v2 test for predict and eval.

--

PiperOrigin-RevId: 249823043

* Add a test enabling get_next_as_optional behavior.

* Remove repeated flag.

* Remove trailing space.

* Make the name shorter.

* Fix lint error.

* Refine the benchmark name.

Adding validation every epoch allows us to view the progress during training instead of having to wait until the last eval. Mostly useful for manual runs.

Current batch size 160000 does not converge to the desired HR. So we decrease to 99k which is known to converge. Tested locally and got to 63.5 at epoch 7. Also decreasing number of epochs as I don't see any improvement after epoch 7-8.

249580533  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

249566870  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Set up BERT benchmark test.

--

PiperOrigin-RevId: 249580533

* Add enable_get_next_as_optional flag.

* Set enable_get_next_as_optional to strategy.

* Add comments to explain the flag.

* Remove trailing whitespace.

* Remove trailing space.

249500988  by hongkuny<hongkuny@google.com>:

    Lints

--

PiperOrigin-RevId: 249500988