GitLab

PiperOrigin-RevId: 263255352

PiperOrigin-RevId: 263230689

PiperOrigin-RevId: 263217684

PiperOrigin-RevId: 263158478

263011341  by hongkuny<hongkuny@google.com>:

    reduce batch size for bert squad v8 accuracy test

--

PiperOrigin-RevId: 263011341

* Replace google3.pyglib modules with tf and absl

This now matches train.py and provides more publicly available libraries.

* Add example pipeline config for SSD Interleaved V2 Model.

Compiled from model_builder_test.py and lstm_ssd_mobilenet_v1_imagenet.config,
Removed data augmentation and tranfer learning (i.e. training from checkpoint) due to errors I was seeing when trying to run with it.

* Remove unused tfrecord creation.

This was also incorrectly specified, as the keys differed from the TFSequenceExample parser.

* correct key specified in docstring

* add tflite frozen graph exporter (cli and lib).

* add tflite model exporter

* add script to test the tflite model

* add mode export documentation

* correct docstring

* rename export files to be unique across detection research work

* correct number of channels for grayscale

* add and correct copyright

262988559  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Enable NCF TF 2.0 model to run on TPUStrategy.

--
262971756  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

262967691  by hongkuny<hongkuny@google.com>:

    Internal

--

PiperOrigin-RevId: 262988559

262962783  by hongkuny<hongkuny@google.com>:

    Internal change

262460803  by hongkuny<hongkuny@google.com>:

    Add a public method to extract shareable layers with decoder.

--
262315011  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Refactor tpu initialization logic to common module.

--
262299019  by akuegel<akuegel@google.com>:

    Internal change

262178259  by hongkuny<hongkuny@google.com>:

    We should call training=True in CTL train step.

--
262081759  by akuegel<akuegel@google.com>:

    Internal change

262021128  by isaprykin<isaprykin@google.com>:

    Internal change

262004398  by taylorrobie<taylorrobie@google.com>:

    Internal change

261786323  by yanhuasun<yanhuasun@google.com>:

    Replace set, dict with ObjectIdentityDict/Set to prepare for eq implementation

--
261393597  by hongkuny<hongkuny@google.com>:

    add an encoder mode for BertModel which returns all layers.

--
261218818  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

261202754  by hongkuny<hongkuny@google.com>:

    Use enable_xla flag for classifier and squad, so xla option is exposed to users.

--
261171038  by gjn<gjn@google.com>:

    Remove weight_decay_rate 0 early exit check

    Removing this code path should be fine since this was actually not doing
    what it meant to do. Since weight_decay_rate is actually a tensor, the
    equality check was only looking at the id of the object and comparing to
    0. This should never be true. Evaluating a tensor is also not what we
    want to do at this point of the code. Thus it should be fine to simply
    remove this code.

--
261169862  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261153520  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261140302  by hongkuny<hongkuny@google.com>:

    Clean up

--
260862396  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Fix BERT pretraining input pipeline to shuffle and shard dataset properly for multi-worker training.

--
260601376  by hongkuny<hongkuny@google.com>:

    reorder Q,K to make TPU faster.

--
260580119  by hongkuny<hongkuny@google.com>:

    Adds expect_partial()

--
260228553  by priyag<priyag@google.com>:

    Enable transformer and NCF official model tests. Also fix some minor issues so that all tests pass with TF 1 + enable_v2_behavior.

--
260060237  by zongweiz<zongweiz@google.com>:

    [BERT SQuAD] Enable mixed precision training

    Add mixed precision training support for BERT SQuAD model. Using the experimental Keras mixed precision API. For numeric stability, use fp32 for layer normalization, dense layers with GELU activation, etc.

--
260052674  by hongkuny<hongkuny@google.com>:

    Add expect_partial()

--
259889221  by hongkuny<hongkuny@google.com>:

    Add no ds / xla / eager perfzero tests

--
259790197  by hongkuny<hongkuny@google.com>:

    Update pretraining model to match tf1 var names.

--
259656389  by hongkuny<hongkuny@google.com>:

    Internal change

259649972  by hongkuny<hongkuny@google.com>:

    Update docs.

--
259470074  by hongkuny<hongkuny@google.com>:

    Adds a dedup phase for trainable variables.

--
259442882  by hongkuny<hongkuny@google.com>:

    Internal

--
259341546  by mrry<mrry@google.com>:

    Remove DEBUG-level logging from the BERT benchmark.

    This triggers graph serialization and other verbose logging in the TensorFlow runtime, which inflates the execution time.

--
259253185  by hongkuny<hongkuny@google.com>:

    Writes a separated checkpoint for the core model in pretraining.
    Clean up export utils to just take a model as argument.

--
258893811  by hongkuny<hongkuny@google.com>:

    Adds summaries for metrics, allowing metrics inside keras.model.

--
258881002  by hongkuny<hongkuny@google.com>:

    Fix lint.

--
258871624  by hongkuny<hongkuny@google.com>:

    Internal change

258597234  by rxsang<rxsang@google.com>:

    Update all the TPUStrategy examples to use the new v2 APIs, i.e.
    make_dataset_iterator -> experimental_distribute_dataset,
    make_input_fn_iterator -> experimental_distribute_datasets_from_function,
    unwrap -> experimental_local_results,
    experimental_run -> experimental_run_v2

--
258581998  by taylorrobie<taylorrobie@google.com>:

    Update keras v2 optimizers to reuse coefficients which are shared across all updates, which reduces the total number of ops created by between 5% (for simple optimizers such as SGD and Adagrad) and 25% (for complicated optimizers such as Adam and NAdam). Separate copies are made for each device and dtype.

    The effect of this change on run time is fairly minimal since Grappler is expected to consolidate most of these ops; however it does improve graph construction time.

--
258208153  by hongkuny<hongkuny@google.com>:

    Adds run_eagerly option for bert.

--
257883986  by hongkuny<hongkuny@google.com>:

    Adds tf.summary for bert training

--
257285772  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

256242827  by yuefengz<yuefengz@google.com>:

    Internal change

256204636  by hongkuny<hongkuny@google.com>:

    Internal

--
256079834  by hongkuny<hongkuny@google.com>:

    Clean up: move common flags together for further refactoring
    Enable steps_per_loop option for all applications.

--
255493073  by hongkuny<hongkuny@google.com>:

    BERT initial OSS readme update.

--
255470372  by dmchen<dmchen@google.com>:

    Slightly expand expected range for F1 score in BERT SQuAD accuracy test

--
255109240  by hongkuny<hongkuny@google.com>:

    Update eval/predict batch sizes.

--
255010016  by hongkuny<hongkuny@google.com>:

    Internal

--
254874613  by hongkuny<hongkuny@google.com>:

    Update glue tasks enum to match directory name

--
254866171  by taylorrobie<taylorrobie@google.com>:

    Internal change

254785517  by zongweiz<zongweiz@google.com>:

    Use train_single_step for BERT GPU models to temporarily work around some performance bugs in GPU runs

--
254497647  by hongkuny<hongkuny@google.com>:

    Fix device placement for TPU export model.

--
254293763  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

254134531  by yuefengz<yuefengz@google.com>:

    Fix a typo in bert_benchmark.py

--
254069984  by hongkuny<hongkuny@google.com>:
    Automated rollback of changelist 254060732.

254061429  by hongkuny<hongkuny@google.com>:

    Use host while loop for training steps.

--
254060732  by yifeif<yifeif@google.com>:
    Automated rollback of changelist 254027750.

254027750  by hongkuny<hongkuny@google.com>:

    Internal change

253850824  by hongkuny<hongkuny@google.com>:

    Improve bert training utils.

--
253818191  by hongkuny<hongkuny@google.com>:

    Update savedmodel export to use new model.save() api.

--
253636854  by dmchen<dmchen@google.com>:

    Run only training in BERT SQuAD performance test

--
253118910  by hongkuny<hongkuny@google.com>:

    Internal change

253113801  by zongweiz<zongweiz@google.com>:

    Internal change

252697519  by dmchen<dmchen@google.com>:

    BERT SQuAD accuracy test

--
252663512  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

--
252647871  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Enable multi worker TPU training for BERT pretraining.

--
252550871  by hongkuny<hongkuny@google.com>:

    Internal change

252522861  by hongkuny<hongkuny@google.com>:

    Remove export using trained model due to implementation error

--
252156812  by yuefengz<yuefengz@google.com>:

    Fix the callback method name in BERT: replaced on_batch_start with on_batch_begin. Without the fix, it won't work with Keras callbacks.

--
251782065  by dmchen<dmchen@google.com>:

    Internal change

251681245  by hongkuny<hongkuny@google.com>:

    Update bert to use the new tf.distribute APIs

--
251575972  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Remove `steps_per_run` when instantiating TPUStrategy.

--
251325964  by hongkuny<hongkuny@google.com>:

    Improve flags

--
251303452  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

250942274  by tobyboyd<tobyboyd@google.com>:

    Internal change

250779087  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Reduce BERT Perfzero benchmark test training steps.

--
250713045  by hongkuny<hongkuny@google.com>:

    TPU util

--
250606180  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Fix BERT benchamrk test errors.

--
250589623  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Change BERT benchmark test pretrained checkpoint url.

--
250587892  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Fix error in BERT custom training loop checkpoint restoration.

--
250577163  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Add logic to inject callback that measures performance in BERT custom training
    loop.

--
250529526  by hongkuny<hongkuny@google.com>:

    Internal clean up

--
250428976  by hongkuny<hongkuny@google.com>:

    Internal change

250415383  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Add min/max value to BERT classifier benchmark test.

--
250376246  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Add benchmark performance test to run BERT on multiple numbers of GPUs.

--
250347237  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Fix linting errors in BERT benchmark test.

--
250326131  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

250315593  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

250303528  by haoyuzhang<haoyuzhang@google.com>:

    Add method docstring to fix lint error.

--
250009207  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Add feature in BERT to write training metrics to a summary file.

--
249896208  by hongkuny<hongkuny@google.com>:

    Adds __init__.py

--
249883771  by hongkuny<hongkuny@google.com>:

    Creates a benchmark dir

--
249580533  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

249566870  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Set up BERT benchmark test.

--
249500988  by hongkuny<hongkuny@google.com>:

    Lints

--
249377254  by hongkuny<hongkuny@google.com>:

    Internal change

249373328  by hongkuny<hongkuny@google.com>:

    Clean up tf import

--
249333938  by hongkuny<hongkuny@google.com>:

    Fix tf1 import

--
249325089  by hongkuny<hongkuny@google.com>:

    BERT 2.0

--
249195008  by tianlin<tianlin@google.com>:

    Internal change

249173564  by hongkuny<hongkuny@google.com>:

    Internal change

246677582  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

245821839  by shiningsun<shiningsun@google.com>:

    Internal change

245353681  by gjn<gjn@google.com>:

    Internal change

245340898  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

245155641  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

244019160  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

242930998  by shiningsun<shiningsun@google.com>:

    Internal change

242049350  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

241663771  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

241054800  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

241028555  by yuefengz<yuefengz@google.com>:

    Internal change

239316550  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

238251867  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

237876559  by taylorrobie<taylorrobie@google.com>:

    Internal change

236346619  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

236182665  by tayo<tayo@google.com>:

    Internal change

234652747  by wangtz<wangtz@google.com>:

    Internal change

233837502  by shiningsun<shiningsun@google.com>:

    Internal change

232033015  by shiningsun<shiningsun@google.com>:

    Internal change

228564809  by taylorrobie<taylorrobie@google.com>:

    Internal change

227052580  by shiningsun<shiningsun@google.com>:

    Internal change

225436264  by shiningsun<shiningsun@google.com>:

    Internal change

222283824  by taylorrobie<taylorrobie@google.com>:

    Internal change

219241224  by taylorrobie<taylorrobie@google.com>:

    Internal change

218774474  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

218610966  by taylorrobie<taylorrobie@google.com>:

    Internal change

218576353  by taylorrobie<taylorrobie@google.com>:

    Internal change

217776707  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

217749789  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

214516790  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

212339556  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

210658133  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

206866123  by taylorrobie<taylorrobie@google.com>:

    Internal change

205252141  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

202519641  by scottzhu<scottzhu@google.com>:

    Internal change

201299684  by kathywu<kathywu@google.com>:

    Internal change

199655516  by karmel<karmel@google.com>:

    Internal change

199209802  by karmel<karmel@google.com>:

    Internal change

198089630  by karmel<karmel@google.com>:

    Internal change

198060863  by karmel<karmel@google.com>:
    Automated rollback of changelist 197920496.

197920496  by kathywu<kathywu@google.com>:

    Internal change

197841416  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

195867348  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

195725348  by taylorrobie<taylorrobie@google.com>:

    Internal change

195283704  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

194662698  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

194103064  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

193581866  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

192783651  by scottzhu<scottzhu@google.com>:
    Automated rollback of changelist 192714881.

192714881  by scottzhu<scottzhu@google.com>:
    Automated rollback of changelist 192710755.

192710755  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

192374551  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

192346754  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

192298443  by karmel<karmel@google.com>:

    Internal change

192220576  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

191514106  by scottzhu<scottzhu@google.com>:

    Internal change

191327699  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

190938103  by karmel<karmel@google.com>:

    Internal change

190804388  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

190479716  by karmel<karmel@google.com>:

    Internal change

189844661  by scottzhu<scottzhu@google.com>:
    Automated rollback of changelist 189816818.

189816818  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

189639056  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

189628781  by karmel<karmel@google.com>:

    Internal change

189267175  by karmel<karmel@google.com>:

    Internal change

189096159  by karmel<karmel@google.com>:

    Internal change

189085341  by karmel<karmel@google.com>:

    Internal change

188949700  by karmel<karmel@google.com>:

    Internal change

PiperOrigin-RevId: 262962783

* Add run_eagerly for ctl.

* fix test name and do not set "default".

Also, do Transformer inference in fp16, as well as training, when --dtype=fp16. In TF 2, layers now cannot run in multiple different dtypes, so we must use the same dtype for training and inference.

262178259  by hongkuny<hongkuny@google.com>:

    We should call training=True in CTL train step.

--
262081759  by akuegel<akuegel@google.com>:

    Internal change

PiperOrigin-RevId: 262178259

* fix steps for 8 gpu tests

262039434  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

262024241  by hongkuny<hongkuny@google.com>:

    Adds __init__.py

--
262021128  by isaprykin<isaprykin@google.com>:

    Internal change

PiperOrigin-RevId: 262039434

262004398  by taylorrobie<taylorrobie@google.com>:

    Internal change

PiperOrigin-RevId: 262004398

* add non-CuDNN LSTM

* condition eager on v2 for no cudnn benchmark

* remove implementation specification

* add comments and adjust benchmarks

* re-add implementation=2

* update docstrings

261786323  by yanhuasun<yanhuasun@google.com>:

    Replace set, dict with ObjectIdentityDict/Set to prepare for eq implementation

--

PiperOrigin-RevId: 261786323

* force_v2_in_keras_compile FLAG default to None and added seperate temp path.

* switch to force testing 1v path not force v2 path.

* Rename function force_v1_path.

* Force GPU placement for Resnet pure eager benchmark

* Create tweaked version of eager benchmark with explicit GPU placement

* Fix the ValueError: Error when checking model input on the new codepath

Fixes the following error:

  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training.py", line 2428, in _standardize_user_data
    exception_prefix='input')
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_utils.py", line 530, in standardize_input_data
    str(len(data)) + ' arrays: ' + str(data)[:200] + '...')
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays: [<tf.Tensor 'cond_8/Identity:0' shape=(None, None) dtype=int64>]..

Tested and reproduced by running trasformer_main_test (thanks to whoever wrote it, phew!)

* Remove the unnecessary TODO.

261393597  by hongkuny<hongkuny@google.com>:

    add an encoder mode for BertModel which returns all layers.

--

PiperOrigin-RevId: 261393597

* Merged commit includes the following changes:
261380794  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261374439  by haoyuzhang<haoyuzhang@google.com>:

    Change Keras CTL dependencies from r1 ResNet to Keras ResNet.

--

PiperOrigin-RevId: 261380794

* Revert unintentional change

* Revert unintentional change

* Update to use py3 lint

* Update model_saving_utils.py

Testing. To be reverted

* Update model_saving_utils.py

old lint is no longer used.

261339941  by haoyuzhang<haoyuzhang@google.com>:

    Own library functions in Keras ResNet models, and remove dependencies on v1 Estimator version of ResNet models.

    Most dependencies that the Keras version has are related to data input pipelines. Created dedicated files (cifar_preprocessing.py, imagenet_preprocessing.py) to collect all logic handling Cifar and ImageNet data input function.

--
261339166  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261317601  by akuegel<akuegel@google.com>:

    Internal change

261218818  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

PiperOrigin-RevId: 261339941

261196859  by yongzhe:

    Integrate EdgeTPU API into the Mobile SSD tflite client.

    Build command with EdgeTPU enabled:
    bazel build mobile_ssd_tflite_client  --define enable_edgetpu=true

    Build command with EdgeTPU disabled:
    bazel build mobile_ssd_tflite_client

--
259096620  by Menglong Zhu:

    Remove unused proto imports.

--

PiperOrigin-RevId: 261196859

261202754  by hongkuny<hongkuny@google.com>:

    Use enable_xla flag for classifier and squad, so xla option is exposed to users.

--

PiperOrigin-RevId: 261202754

* Restructure resnet estimator code to under official/r1

* Continue moving resnet code...

* Improved README.md

261171038  by gjn<gjn@google.com>:

    Remove weight_decay_rate 0 early exit check

    Removing this code path should be fine since this was actually not doing
    what it meant to do. Since weight_decay_rate is actually a tensor, the
    equality check was only looking at the id of the object and comparing to
    0. This should never be true. Evaluating a tensor is also not what we
    want to do at this point of the code. Thus it should be fine to simply
    remove this code.

--
261169862  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261153520  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261140302  by hongkuny<hongkuny@google.com>:

    Clean up

--

PiperOrigin-RevId: 261171038

260862396  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Fix BERT pretraining input pipeline to shuffle and shard dataset properly for multi-worker training.

--

PiperOrigin-RevId: 260862396

* Merged commit includes the following changes:
260800447  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Fix tf function signature errors for NCF model.

--
260767064  by psv<psv@google.com>:

    Renames the `run_distributed` flag to `experimental_run_tf_function` and modifies the tests generated by the keras_all_modes decorator to better reflect that.

--

PiperOrigin-RevId: 260800447

* Update ncf_keras_benchmark.py

* Update ncf_keras_benchmark.py

* Update ncf_keras_benchmark.py

* Update ncf_keras_benchmark.py