GitLab

PiperOrigin-RevId: 368957441

PiperOrigin-RevId: 368935233

PiperOrigin-RevId: 368868424

PiperOrigin-RevId: 368778443

PiperOrigin-RevId: 368573031

PiperOrigin-RevId: 368571725

PiperOrigin-RevId: 368535973

PiperOrigin-RevId: 368535232

This CL changes ResidualBlock and InvertedBottleneckBlock.

PiperOrigin-RevId: 368383954

PiperOrigin-RevId: 368260712

PiperOrigin-RevId: 368257425

PiperOrigin-RevId: 368157180

PiperOrigin-RevId: 368130039

PiperOrigin-RevId: 368129317

This replaces symbols in tf.keras.mixed_precision.experimental with the corresponding nonexperimental symbols. In some cases, passing a Policy is replaced with passing a policy name for conciseness.

Additionally, for the Shakespeare model, the loss_scale flag is removed, since supporting it with the nonexperimental API is slightly more verbose and it is recommended users use the default loss scale.

PiperOrigin-RevId: 368123944

PiperOrigin-RevId: 368122127

For all modified calls to set_mixed_precision_policy(), the loss_scale argument was removed, as it cannot be passed if the nonexperimental API is used. For all such callers, the loss_scale is later used to explicitly create a LossScaleOptimizer, so removing the argument has no impact.

Switching to the non-experimental LossScaleOptimizer has no effect, as it has near identical behavior and all isinstance checks within the official models check for the non-experimental version.

PiperOrigin-RevId: 368101975

PiperOrigin-RevId: 368070382

PiperOrigin-RevId: 368067415

PiperOrigin-RevId: 368040234

PiperOrigin-RevId: 368036370

All models which support loss scaling support dynamic loss scaling, so the argument has no purpose. It used to be that some models scaled the loss manually instead of using a LossScaleOptimizer, and so did not support dynamic loss scaling.

PiperOrigin-RevId: 367719521

PiperOrigin-RevId: 367679732

PiperOrigin-RevId: 367522154

PiperOrigin-RevId: 367463455

PiperOrigin-RevId: 367244904

PiperOrigin-RevId: 367105004

PiperOrigin-RevId: 367101911

PiperOrigin-RevId: 367083514

This has no functional impact since the default is currently True, but I plan on changing the default to False soon.

PiperOrigin-RevId: 367049942

The function `tf.train.experimental.enable_mixed_precision_graph_rewrite` will be removed from the TF2 namespace soon, at which point it will only be accessible under tf.compat.v1.

PiperOrigin-RevId: 367046393

PiperOrigin-RevId: 366939051

For the `GaussianProcessClassificationHead`, the temperature scaling needs to be disabled during training to avoid unexpected modification to the learning rate, which harms model quality. (Unfortunately, this seems to require adding `training` to the `call` method).

Also set the default of `gp_cov_ridge_penalty` in `RandomFeatureGaussianProcess` to 1 to be consistent with that in the `GaussianProcessClassificationHead`.

PiperOrigin-RevId: 366917075

PiperOrigin-RevId: 366900579

This shouldn't break any official models, since I changed all LossScaleOptimizer isinstance checks to use the nonexperimental version (the experimental LSO subclasses the nonexperimental LSO, so changing isinstance checks in this way is always safe).

PiperOrigin-RevId: 366891847

This change allows `GaussianProcessClassificationHead` to output only the predictive logits during training and evaluation (instead of outputting a tuple  `(logits, covmat)`). The goal is to make the layer more compatible with `SentencePredictionTask` and Keras' `model.fit()` API.

PiperOrigin-RevId: 366891298

PiperOrigin-RevId: 366889393

PiperOrigin-RevId: 366883662

PiperOrigin-RevId: 366883391

PiperOrigin-RevId: 366883351