GitLab

This reverts commit 649e0374.

We've deprecated the "tower" terminology in DistributionStrategy, so the "cross_tower_ops" argument is now "cross_device_ops", matching the current name of "AllReduceCrossDeviceOps".

`num_replicas` is being replaced by `num_replicas_in_sync` and something that returns the number of steps running concurrently.

This tag should match EVAL_HP_NUM_NEG.

I've noticed sometimes the async process's pool processes do not die when ncf_main.py ends and kills the async process. This commit fixes the issue.

* Keras-ify TPU embedding lookup

* delint

* pull get_variable() out of keras lambda

* delint

* move get_variable under variable scope

* Merges TPU-TC optimizations into HEAD.

* Split a line that went over 80 from a tab.

* Remove trailing whitespace.

The option is --nouse_estimator

--ml_perf now just changes the model to make it MLPerf compliant. --output_ml_perf_compliance_logging adds the MLPerf compliance logs.

prevent async process from writing alive file until the main process has created the cache root (#5614)

The error message was:

absl.flags._exceptions.IllegalFlagValueError: flag --ml_perf=None: ('Non-boolean argument to boolean flag', 'None')

* move version check to a function

* delint

* tweak pip check

* delint

To match new terminology in DistributionStrategy.

* first pass at __getattr__ abuse logger

* first pass at adding tags to NCF

* minor formatting updates

* fix tag name

* convert metrics to python floats

* getting closer...

* direct mlperf logs to a file

* small tweaks and add stitching

* update tags

* fix tag and add a sudo call

* tweak format of run.sh

* delint

* use distribution strategies for evaluation

* address PR comments

* delint and fix test

* adjust flag validation for xla

* add prefix to distinguish log stitching

* fix index bug

* fix clear cache for root user

* dockerize cache drop

* TIL some regex magic