GitLab

* Add XLA benchmark tests FP32 only for now.

* Add FP16 XLA tests.

* FP16 only tests.

* set default steps to 300K.

* Log flags to perfzero.

* Add XLA support to transformer

- Moved config logic to keras_utils
- Added enable_xla flag to _performance flags
- Did not refactor enable_xla flag from keras resnet due to
  reliance on calling FLAGs in estimator keras and that is
  a needed refactor for another time.

* fix g3 lint complaint.

* Refactor set config into keras_utils.

* Move flags out of main.

* pipe through enable_xla

* Update official/transformer/v2/misc.py
Co-Authored-By: Reed <reedwm@google.com>

252534787  by hongkuny<hongkuny@google.com>:

    Transformer vocab fix to strip correctly in py2

--

PiperOrigin-RevId: 252534787

251762562  by hongkuny<hongkuny@google.com>:

    Fix blue score inconsistency

--

PiperOrigin-RevId: 251762562

* Fix various lint errors

* Fix logging format

* Make max_length and static_batch configurable.

* Fix line length.

* Fix incorrect parameters in building eval input.

* Improve comments for readability.

* Add static batch benchmarks to estimator 

So we can distinguish how much static vs dynamic batch matter.

* change max_length for static_batch tests

* Add flag for max length

this is not going to help with current tf.data semantics. so removing it.

* Fixes that make transformer run.

* Remove debug print statements.

* Changed the permissions to 644.

* Fix the rest of the permissions.

* enable static batch in all benchmarks

* Restrict dist strat hack to training mode

For now we will do predict/eval without dist strat, so remove that hack in non training cases.

* Use `inputs` instead of `x` as arg name for call

Keras has different behavior based on whether the inputs are called `inputs` or not. Using `inputs` gives expected behaviors.

* Avoid extra map fn on input in dist strat case

* Update how we handle custom metrics

This new approach works with and without dist strat. The previous one didn't work with dist strat. We need to fix that but this is reasonable in meantime (b/133724664).

* Update benchmarks

* typo in metrics code

* Revert metrics change

Didn't actually work in distributed case..

This shuffling should help in getting shuffling each epoch.

* Moved common keras code to utils.

* Initial 1 gpu benchmark

- Aligned flags with resnet example
- removed code/features that are not super useful
- eval as part of train if bleu source/ref provided
- add exp_per_second hook

* Rename benchmark classes, pass batch-size and log_steps.

* fix docstring

* Predict done with checkpoints inline

- perfzero baseclass

* steps not epochs with smoother training loop.

* do not initialize history outside loop.

* 5000 between eval not 500

* estimator to keras.

* remove epochs var.

* use range not xrange.

* 200K steps for 1 gpu

* fix global step

* Merged commit includes the following changes:
249776315  by tianlin<tianlin@google.com>:

    Internal change

249763206  by tianlin<tianlin@google.com>:

    For TF 2.0 (related to Beam Search), expand cond dims in tf.where(cond, x, y) to make all parameters broadcastable.

--
249392724  by hongkuny<hongkuny@google.com>:

    Internal change

PiperOrigin-RevId: 249776315

* Merged commit includes the following changes:
249823043  by tianlin<tianlin@google.com>:

    Bring back v2 test for predict and eval.

--

PiperOrigin-RevId: 249823043

* Add big tests.

* fix super

* Add fp16, increase 8xGPU batch-sizes

* Adding the rest of the fp16 tests.

* Big accuracy test batch_perf_gpu

* fix docstrings

* add _run_and_report

* Edited docstrings

* Merged commit includes the following changes:
249218656  by tianlin<tianlin@google.com>:

    Deal with imports, fix a typo and make unit tests fast.

--
249198645  by tianlin<tianlin@google.com>:

    Trivial: Remove one empty line before "import tensorflow"

--
249195490  by tianlin<tianlin@google.com>:

    Initialize Transformer TF V2 Model with Keras subclassing implementation. (Compatible with TF V1)

--
249195008  by tianlin<tianlin@google.com>:

    Internal change

249173564  by hongkuny<hongkuny@google.com>:

    Internal change

249079258  by hongkuny<hongkuny@google.com>:

    Internal change

247691534  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

247533725  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

247509295  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

247311355  by wangtz<wangtz@google.com>:

    Internal change

247303127  by wangtz<wangtz@google.com>:

  ...

* Add FP16 and benchmarks.

* add missing run and report.

* Add loss_scale as option not included with dtype.

* move loss_scale validation under dtype conditional.

* add loss_scale to flags tested.

* Add first benchmark and return stats.

* Remove print statements update training steps.

* Revert print T: in print statement.

* Remove print(stats)

* add 2 gpu accuracy test for base.

* Fixed total_batch_size when using gpu + gFile deprecations.

* 8 GPU test name fix

* Add 4 and 8 GPU tests.

* typo fixes.

* Clean up test names and methods.

* bleu uncased.  docstring format fix.

Replace per_device with per_replica and PerDevice with PerReplica, because the PerDevice concept was renamed and doesn't exist anymore. (#6693)

* Replace per_device with per_replica and PerDevice with PerReplica, because the PerReplica concept was renamed and doesn't exist anymore.

* Update README.md

* Update README.md

* Update README.md