GitLab

* first pass at __getattr__ abuse logger

* first pass at adding tags to NCF

* minor formatting updates

* fix tag name

* convert metrics to python floats

* getting closer...

* direct mlperf logs to a file

* small tweaks and add stitching

* update tags

* fix tag and add a sudo call

* tweak format of run.sh

* delint

* use distribution strategies for evaluation

* address PR comments

* delint and fix test

* adjust flag validation for xla

* add prefix to distinguish log stitching

* fix index bug

* fix clear cache for root user

* dockerize cache drop

* TIL some regex magic

* intermediate commit

finish replacing spillover with resampled padding

intermediate commit

* resolve merge conflict

* intermediate commit

* further consolidate the data pipeline

* complete first pass at data pipeline refactor

* remove some leftover code

* fix test

* remove resampling, and move train padding logic into neumf.py

* small tweaks

* fix weight bug

* address PR comments

* fix dict zip. (Reed led me astray)

* delint

* make data test deterministic and delint

* Reed didn't lead me astray. I just can't read.

* more delinting

* even more delinting

* use resampling for last batch padding

* pad last batch with unique data

* Revert "pad last batch with unique data"

This reverts commit cbdf46efcd5c7907038a24105b88d38e7f1d6da2.

* move padded batch to the beginning

* delint

* fix step check for synthetic data

Add perf args

Drop references to `is_single_tower`.

Since we plan on deleting this method, it is only used in distribution_utils_test.py.

 Refactor neumf_model.py to support users who just need top_k and ndcg tensors.

Improvements to AstroNet and add AstroWaveNet

PiperOrigin-RevId: 217341274

PiperOrigin-RevId: 215014489

Add AstroWaveNet model, a generative model of astronomical light curves. AstroWaveNet's hidden states can be used in a semi-supervised fashion for downstream tasks like finding exoplanets.

PiperOrigin-RevId: 214073725

PiperOrigin-RevId: 214069349

PiperOrigin-RevId: 213963765

PiperOrigin-RevId: 213353962

PiperOrigin-RevId: 212909744

Factor out evaluate(), continuous_eval(), and continuous_train_and_eval() from estimator_util.py into estimator_runner.py.

PiperOrigin-RevId: 212903406

PiperOrigin-RevId: 211832751

1. reshard_arrays(xs, ys): Reshards arrays in xs to match the lengths of arrays in ys.
2. uniform_cadence_light_curve(): Combines data into a single light curve with uniform cadence numbers.

PiperOrigin-RevId: 211724321

Remove the constant LONG_CADENCE_TIME_DELTA_DAYS as this is not strictly constant between light curves (although it is very close).

PiperOrigin-RevId: 211153560

This applies mainly to scrambled data (NaN time values typically come with NaN flux values, which are removed anyway, but scrambing decouples NaN time values from NaN flux values).

PiperOrigin-RevId: 209029696

Update configurations with label mapping for simulated data and new batch_size and learning_rate from vizier studies.

PiperOrigin-RevId: 208902383

PiperOrigin-RevId: 208863882

PiperOrigin-RevId: 208862798

Ensure consistent argument order, change NaN processing to remove NaN times as well, and do so before scrambling

PiperOrigin-RevId: 207804986

Modify processing pipeline to enable generation of scrambled lightcurves. Fix bugs to enable generation of inverted lightcurves.

PiperOrigin-RevId: 207595688

Inversion should be done *after* fitting a normalization curve to the stellar variability and dividing it away. Doing it before normalizing may result in unintentionally reversing the inversion. Therefore we remove the option to do it at this point.

PiperOrigin-RevId: 207309650

PiperOrigin-RevId: 205427760

PiperOrigin-RevId: 205168785

* move flagfile into the cache_dir

* remove duplicate code

* delint

* Use data_dir instead of flags.FLAGS.data_dir in data_preprocessing.py.

* Use data_dir instead of flags.FLAGS.data_dir in data_preprocessing.py.

* Replace multiprocess pool with popen_helper.get_pool() in data_preprocessing.