GitLab

PiperOrigin-RevId: 276518206

PiperOrigin-RevId: 273966871

Instead of needing to ensure variables are float32, casting inputs to float32, etc, instead dtype="float32" is passed to the layer constructor, which will do all that logic automatically.

The only difference is the output of LayerNorm is now float32 instead of float16, so an extra cast is needed elsewhere.

PiperOrigin-RevId: 273833286

PiperOrigin-RevId: 273371605

PiperOrigin-RevId: 269914707

Refactor basic utils to modeling/

PiperOrigin-RevId: 269600561

PiperOrigin-RevId: 269376599

PiperOrigin-RevId: 268057019

--stop_threshold, --num_gpu, --hooks, --export_dir, and --distribution_strategy have been unexposed from models which do not use them

PiperOrigin-RevId: 268032080

PiperOrigin-RevId: 267435985

--clean, --train_epochs, and --epochs_between_evals have been unexposed from models which do not use them

PiperOrigin-RevId: 267065651

PiperOrigin-RevId: 266848625

I soon plan on making mixed precision policies exposed in TF2 only.

PiperOrigin-RevId: 266433372

PiperOrigin-RevId: 266413847

The transformer model requires TF2.

PiperOrigin-RevId: 266026991

PiperOrigin-RevId: 265994446

PiperOrigin-RevId: 265856020

PiperOrigin-RevId: 265771605

PiperOrigin-RevId: 265499806

PiperOrigin-RevId: 265122309

PiperOrigin-RevId: 264958330

PiperOrigin-RevId: 264867908

PiperOrigin-RevId: 264853703

PiperOrigin-RevId: 264776396

The old infer_float32_policies policy will be removed from TensorFlow soon.

An earlier change made it so running this file would cause an error. The unit tests still passed, as the unit tests do not directly call the main() function

PiperOrigin-RevId: 264268400

Small tweak to the range based on seeing values
slightly over 29 and slightly under 28. This avoids
false positives that even if valid are not actionable.

PiperOrigin-RevId: 264174941

PiperOrigin-RevId: 263863438

PiperOrigin-RevId: 263812316

PiperOrigin-RevId: 263669433

PiperOrigin-RevId: 263485329

PiperOrigin-RevId: 263463300

Also, do Transformer inference in fp16, as well as training, when --dtype=fp16. In TF 2, layers now cannot run in multiple different dtypes, so we must use the same dtype for training and inference.

262039434  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

262024241  by hongkuny<hongkuny@google.com>:

    Adds __init__.py

--
262021128  by isaprykin<isaprykin@google.com>:

    Internal change

PiperOrigin-RevId: 262039434