GitLab

For all modified calls to set_mixed_precision_policy(), the loss_scale argument was removed, as it cannot be passed if the nonexperimental API is used. For all such callers, the loss_scale is later used to explicitly create a LossScaleOptimizer, so removing the argument has no impact.

Switching to the non-experimental LossScaleOptimizer has no effect, as it has near identical behavior and all isinstance checks within the official models check for the non-experimental version.

PiperOrigin-RevId: 368101975

PiperOrigin-RevId: 368070382

PiperOrigin-RevId: 368067415

PiperOrigin-RevId: 368040234

PiperOrigin-RevId: 368036370

Use sparse_categorical_crossentropy for test as the loss object default does not work on tpustrategy +
the single task trainer already handles the reduction.

PiperOrigin-RevId: 367757677

All models which support loss scaling support dynamic loss scaling, so the argument has no purpose. It used to be that some models scaled the loss manually instead of using a LossScaleOptimizer, and so did not support dynamic loss scaling.

PiperOrigin-RevId: 367719521

All models which support loss scaling support dynamic loss scaling, so the argument has no purpose. It used to be that some models scaled the loss manually instead of using a LossScaleOptimizer, and so did not support dynamic loss scaling.

PiperOrigin-RevId: 367719521

PiperOrigin-RevId: 367679732

PiperOrigin-RevId: 367564187

PiperOrigin-RevId: 367522154

PiperOrigin-RevId: 367463455

PiperOrigin-RevId: 367244904

PiperOrigin-RevId: 367105004

PiperOrigin-RevId: 367101911

PiperOrigin-RevId: 367083514

This has no functional impact since the default is currently True, but I plan on changing the default to False soon.

PiperOrigin-RevId: 367049942

The function `tf.train.experimental.enable_mixed_precision_graph_rewrite` will be removed from the TF2 namespace soon, at which point it will only be accessible under tf.compat.v1.

PiperOrigin-RevId: 367046393

The function `tf.train.experimental.enable_mixed_precision_graph_rewrite` will be removed from the TF2 namespace soon, at which point it will only be accessible under tf.compat.v1.

PiperOrigin-RevId: 367046393

PiperOrigin-RevId: 366939051

For the `GaussianProcessClassificationHead`, the temperature scaling needs to be disabled during training to avoid unexpected modification to the learning rate, which harms model quality. (Unfortunately, this seems to require adding `training` to the `call` method).

Also set the default of `gp_cov_ridge_penalty` in `RandomFeatureGaussianProcess` to 1 to be consistent with that in the `GaussianProcessClassificationHead`.

PiperOrigin-RevId: 366917075

PiperOrigin-RevId: 366900579

This shouldn't break any official models, since I changed all LossScaleOptimizer isinstance checks to use the nonexperimental version (the experimental LSO subclasses the nonexperimental LSO, so changing isinstance checks in this way is always safe).

PiperOrigin-RevId: 366891847

This change allows `GaussianProcessClassificationHead` to output only the predictive logits during training and evaluation (instead of outputting a tuple  `(logits, covmat)`). The goal is to make the layer more compatible with `SentencePredictionTask` and Keras' `model.fit()` API.

PiperOrigin-RevId: 366891298

PiperOrigin-RevId: 366889393

PiperOrigin-RevId: 366883662

PiperOrigin-RevId: 366883391

PiperOrigin-RevId: 366883351

PiperOrigin-RevId: 366879385

PiperOrigin-RevId: 366819679

PiperOrigin-RevId: 366817283

PiperOrigin-RevId: 366540340

PiperOrigin-RevId: 366496850

PiperOrigin-RevId: 366395416

PiperOrigin-RevId: 366391425

PiperOrigin-RevId: 366382220

PiperOrigin-RevId: 366368802

PiperOrigin-RevId: 366354480

PiperOrigin-RevId: 366330392

PiperOrigin-RevId: 366313554