GitLab

Remove progress indicator from Movielens download

update the calculation of num_batches_per_epoch

the number of batches per epoch also depends on the number of gpus. it should be `num_batches_per_epoch = (cifar10.NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN / FLAGS.batch_size / FLAGS.num_gpus)`

* Keras-ify TPU embedding lookup

* delint

* pull get_variable() out of keras lambda

* delint

* move get_variable under variable scope

* Merges TPU-TC optimizations into HEAD.

* Split a line that went over 80 from a tab.

* Remove trailing whitespace.

The option is --nouse_estimator

--ml_perf now just changes the model to make it MLPerf compliant. --output_ml_perf_compliance_logging adds the MLPerf compliance logs.

prevent async process from writing alive file until the main process has created the cache root (#5614)

AllReduceCrossTowerOps -> AllReduceCrossDeviceOps

The error message was:

absl.flags._exceptions.IllegalFlagValueError: flag --ml_perf=None: ('Non-boolean argument to boolean flag', 'None')

* move version check to a function

* delint

* tweak pip check

* delint

To match new terminology in DistributionStrategy.

* first pass at __getattr__ abuse logger

* first pass at adding tags to NCF

* minor formatting updates

* fix tag name

* convert metrics to python floats

* getting closer...

* direct mlperf logs to a file

* small tweaks and add stitching

* update tags

* fix tag and add a sudo call

* tweak format of run.sh

* delint

* use distribution strategies for evaluation

* address PR comments

* delint and fix test

* adjust flag validation for xla

* add prefix to distinguish log stitching

* fix index bug

* fix clear cache for root user

* dockerize cache drop

* TIL some regex magic

* intermediate commit

finish replacing spillover with resampled padding

intermediate commit

* resolve merge conflict

* intermediate commit

* further consolidate the data pipeline

* complete first pass at data pipeline refactor

* remove some leftover code

* fix test

* remove resampling, and move train padding logic into neumf.py

* small tweaks

* fix weight bug

* address PR comments

* fix dict zip. (Reed led me astray)

* delint

* make data test deterministic and delint

* Reed didn't lead me astray. I just can't read.

* more delinting

* even more delinting

* use resampling for last batch padding

* pad last batch with unique data

* Revert "pad last batch with unique data"

This reverts commit cbdf46efcd5c7907038a24105b88d38e7f1d6da2.

* move padded batch to the beginning

* delint

* fix step check for synthetic data

Add perf args

Drop references to `is_single_tower`.

Since we plan on deleting this method, it is only used in distribution_utils_test.py.

 Refactor neumf_model.py to support users who just need top_k and ndcg tensors.

Improvements to AstroNet and add AstroWaveNet

PiperOrigin-RevId: 217341274

PiperOrigin-RevId: 215014489

Add AstroWaveNet model, a generative model of astronomical light curves. AstroWaveNet's hidden states can be used in a semi-supervised fashion for downstream tasks like finding exoplanets.

PiperOrigin-RevId: 214073725

PiperOrigin-RevId: 214069349

PiperOrigin-RevId: 213963765

PiperOrigin-RevId: 213353962

PiperOrigin-RevId: 212909744

Factor out evaluate(), continuous_eval(), and continuous_train_and_eval() from estimator_util.py into estimator_runner.py.

PiperOrigin-RevId: 212903406

PiperOrigin-RevId: 211832751

1. reshard_arrays(xs, ys): Reshards arrays in xs to match the lengths of arrays in ys.
2. uniform_cadence_light_curve(): Combines data into a single light curve with uniform cadence numbers.

PiperOrigin-RevId: 211724321

Remove the constant LONG_CADENCE_TIME_DELTA_DAYS as this is not strictly constant between light curves (although it is very close).

PiperOrigin-RevId: 211153560