GitLab

PiperOrigin-RevId: 384478988

Before, it was too easy to accidentally forget to set runtime.loss_scale, which had to always be done if mixed precision is used, otherwise the model would converge to worse accuracy. Now, all that needs to be done to use mixed precision is to set runtime.mixed_precision_dtype=float16.

PiperOrigin-RevId: 383767033

Adds a feature to process a dictionary for `input_paths` in `DataConfig`, to allow combining multiple datasets using a user defined combine_fn.

PiperOrigin-RevId: 381363688

In nlp/train.py and vision/beta/train.py, certain flags are marked as required. Additionally, in certain functions, error messages are improved if a necessary flag is not specified, which is a fallback in case a file calling define_flags() does not mark the necessary flags are required. Previously if any of these flags were not specified, it would crash with a cryptic error message, making it hard to tell what went wrong.

In a subsequent change, I will mark flags as required in more files which call define_flags().

PiperOrigin-RevId: 381066985

PiperOrigin-RevId: 380858205

PiperOrigin-RevId: 380470496

PiperOrigin-RevId: 379618127

PiperOrigin-RevId: 378899878

PiperOrigin-RevId: 376928064

PiperOrigin-RevId: 376298243

PiperOrigin-RevId: 374244811

PiperOrigin-RevId: 373725548

PiperOrigin-RevId: 373623867

PiperOrigin-RevId: 372242256

PiperOrigin-RevId: 368778443

PiperOrigin-RevId: 368257425

PiperOrigin-RevId: 368122127

For all modified calls to set_mixed_precision_policy(), the loss_scale argument was removed, as it cannot be passed if the nonexperimental API is used. For all such callers, the loss_scale is later used to explicitly create a LossScaleOptimizer, so removing the argument has no impact.

Switching to the non-experimental LossScaleOptimizer has no effect, as it has near identical behavior and all isinstance checks within the official models check for the non-experimental version.

PiperOrigin-RevId: 368101975

This shouldn't break any official models, since I changed all LossScaleOptimizer isinstance checks to use the nonexperimental version (the experimental LSO subclasses the nonexperimental LSO, so changing isinstance checks in this way is always safe).

PiperOrigin-RevId: 366891847

PiperOrigin-RevId: 366879385

PiperOrigin-RevId: 366819679

PiperOrigin-RevId: 366368802

PiperOrigin-RevId: 366330392

PiperOrigin-RevId: 364734183

PiperOrigin-RevId: 363782489

PiperOrigin-RevId: 363595377

PiperOrigin-RevId: 363529035

PiperOrigin-RevId: 362729857

The default is True, but I plan on changing it to False soon. After that, I plan on removing the argument and never using the experimental API.

PiperOrigin-RevId: 360724698

PiperOrigin-RevId: 360539689

PiperOrigin-RevId: 360344113

PiperOrigin-RevId: 360262681

PiperOrigin-RevId: 360256877

https://www.tensorflow.org/datasets/api_docs/python/tfds/load

PiperOrigin-RevId: 360237231

PiperOrigin-RevId: 358727239

PiperOrigin-RevId: 358289482

[core] Update BestCheckpointExporter to support dictionary with arbitrary depth (possibly coming from multitask eval).
Also added best checkpointexporter support to multitask.

PiperOrigin-RevId: 357584596

PiperOrigin-RevId: 357078424

- Create shadow weights at the beginning of training.
- Swap the weights during.
- Save best checkpoint with average weights.

The following fields need to be set in order to activate the best checkpoint exporter.
best_checkpoint_eval_metric
best_checkpoint_export_subdir
best_checkpoint_metric_comp

To serve, or to finetune the trained checkpoints on a target dataset, use checkpoints under best_checkpoint_export_subdir.

PiperOrigin-RevId: 356093831

PiperOrigin-RevId: 355082063