DELF documentation updates (#8767)

* Merged commit includes the following changes:
253126424  by Andre Araujo:

    Scripts to compute metrics for Google Landmarks dataset.

    Also, a small fix to metric in retrieval case: avoids duplicate predicted images.

--
253118971  by Andre Araujo:

    Metrics for Google Landmarks dataset.

--
253106953  by Andre Araujo:

    Library to read files from Google Landmarks challenges.

--
250700636  by Andre Araujo:

    Handle case of aggregation extraction with empty set of input features.

--
250516819  by Andre Araujo:

    Add minimum size for DELF extractor.

--
250435822  by Andre Araujo:

    Add max_image_size/min_image_size for open-source DELF proto / module.

--
250414606  by Andre Araujo:

    Refactor extract_aggregation to allow reuse with different datasets.

--
250356863  by Andre Araujo:

    Remove unnecessary cmd_args variable from boxes_and_features_extraction.

--
249783379  by Andre Araujo:

    Create directory for writing mapping file if it does not exist.

--
249581591  by Andre Araujo:

    Refactor scripts to extract boxes and features from images in Revisited datasets.
    Also, change tf.logging.info --> print for easier logging in open source code.

--
249511821  by Andre Araujo:

    Small change to function for file/directory handling.

--
249289499  by Andre Araujo:

    Internal change.

--

PiperOrigin-RevId: 253126424

* Updating DELF init to adjust to latest changes

* Editing init files for python packages

* Edit D2R dataset reader to work with py3.

PiperOrigin-RevId: 253135576

* DELF package: fix import ordering

* Adding new requirements to setup.py

* Adding init file for training dir

* Merged commit includes the following changes:

FolderOrigin-RevId: /google/src/cloud/andrearaujo/delf_oss/google3/..

* Adding init file for training subdirs

* Working version of DELF training

* Internal change.

PiperOrigin-RevId: 253248648

* Fix variance loading in open-source code.

PiperOrigin-RevId: 260619120

* Separate image re-ranking as a standalone library, and add metric writing to dataset library.

PiperOrigin-RevId: 260998608

* Tool to read written D2R Revisited datasets metrics file. Test is added.

Also adds a unit test for previously-existing SaveMetricsFile function.

PiperOrigin-RevId: 263361410

* Add optional resize factor for feature extraction.

PiperOrigin-RevId: 264437080

* Fix NumPy's new version spacing changes.

PiperOrigin-RevId: 265127245

* Maker image matching function visible, and add support for RANSAC seed.

PiperOrigin-RevId: 277177468

* Avoid matplotlib failure due to missing display backend.

PiperOrigin-RevId: 287316435

* Removes tf.contrib dependency.

PiperOrigin-RevId: 288842237

* Fix tf contrib removal for feature_aggregation_extractor.

PiperOrigin-RevId: 289487669

* Merged commit includes the following changes:
309118395  by Andre Araujo:

    Make DELF open-source code compatible with TF2.

--
309067582  by Andre Araujo:

    Handle image resizing rounding properly for python extraction.

    New behavior is tested with unit tests.

--
308690144  by Andre Araujo:

    Several changes to improve DELF model/training code and make it work in TF 2.1.0:
    - Rename some files for better clarity
    - Using compat.v1 versions of functions
    - Formatting changes
    - Using more appropriate TF function names

--
308689397  by Andre Araujo:

    Internal change.

--
308341315  by Andre Araujo:

    Remove old slim dependency in DELF open-source model.

    This avoids issues with requiring old TF-v1, making it compatible with latest TF.

--
306777559  by Andre Araujo:

    Internal change

--
304505811  by Andre Araujo:

    Raise error during geometric verification if local features have different dimensionalities.

--
301739992  by Andre Araujo:

    Transform some geometric verification constants into arguments, to allow custom matching.

--
301300324  by Andre Araujo:

    Apply name change(experimental_run_v2 -> run) for all callers in Tensorflow.

--
299919057  by Andre Araujo:

    Automated refactoring to make code Python 3 compatible.

--
297953698  by Andre Araujo:

    Explicitly replace "import tensorflow" with "tensorflow.compat.v1" for TF2.x migration

--
297521242  by Andre Araujo:

    Explicitly replace "import tensorflow" with "tensorflow.compat.v1" for TF2.x migration

--
297278247  by Andre Araujo:

    Explicitly replace "import tensorflow" with "tensorflow.compat.v1" for TF2.x migration

--
297270405  by Andre Araujo:

    Explicitly replace "import tensorflow" with "tensorflow.compat.v1" for TF2.x migration

--
297238741  by Andre Araujo:

    Explicitly replace "import tensorflow" with "tensorflow.compat.v1" for TF2.x migration

--
297108605  by Andre Araujo:

    Explicitly replace "import tensorflow" with "tensorflow.compat.v1" for TF2.x migration

--
294676131  by Andre Araujo:

    Add option to resize images to square resolutions without aspect ratio preservation.

--
293849641  by Andre Araujo:

    Internal change.

--
293840896  by Andre Araujo:

    Changing Slim import to tf_slim codebase.

--
293661660  by Andre Araujo:

    Allow the delf training script to read from TFRecords dataset.

--
291755295  by Andre Araujo:

    Internal change.

--
291448508  by Andre Araujo:

    Internal change.

--
291414459  by Andre Araujo:

    Adding train script.

--
291384336  by Andre Araujo:

    Adding model export script and test.

--
291260565  by Andre Araujo:

    Adding placeholder for Google Landmarks dataset.

--
291205548  by Andre Araujo:

    Definition of DELF model using Keras ResNet50 as backbone.

--
289500793  by Andre Araujo:

    Add TFRecord building script for delf.

--

PiperOrigin-RevId: 309118395

* Updating README, dependency versions

* Updating training README

* Fixing init import of export_model

* Fixing init import of export_model_utils

* tkinter in INSTALL_INSTRUCTIONS

* Merged commit includes the following changes:

FolderOrigin-RevId: /google/src/cloud/andrearaujo/delf_oss/google3/..

* INSTALL_INSTRUCTIONS mentioning different cloning options

* Updating required TF version, since 2.1 is not available in pip

* Internal change.

PiperOrigin-RevId: 309136003

* Fix missing string_input_producer and start_queue_runners in TF2.

PiperOrigin-RevId: 309437512

* Handle RANSAC from skimage's latest versions.

PiperOrigin-RevId: 310170897

* DELF 2.1 version: badge and setup.py updated

* Add TF version badge in INSTALL_INSTRUCTIONS and paper badges in README

* Add paper badges in paper instructions

* Add paper badge to landmark detection instructions

* Small update to DELF training README

* Merged commit includes the following changes:
312614961  by Andre Araujo:

    Instructions/code to reproduce DELG paper results.

--
312523414  by Andre Araujo:

    Fix a minor bug when post-process extracted features, format config.delf_global_config.image_scales_ind to a list.

--
312340276  by Andre Araujo:

    Add support for global feature extraction in DELF open-source codebase.

--
311031367  by Andre Araujo:

    Add use_square_images as an option in DELF config. The default value is false. if it is set, then images are resized to square resolution before feature extraction (e.g. Starburst use case. ) Thought for a while, whether to have two constructor of DescriptorToImageTemplate, but in the end, decide to only keep one, may be less confusing.

--
310658638  by Andre Araujo:

    Option for producing local feature-based image match visualization.

--

PiperOrigin-RevId: 312614961

* DELF README update / DELG instructions

* DELF README update

* DELG instructions update

* Merged commit includes the following changes:

PiperOrigin-RevId: 312695597

* Merged commit includes the following changes:
312754894  by Andre Araujo:

    Code edits / instructions to reproduce GLDv2 results.

--

PiperOrigin-RevId: 312754894

* Markdown updates after adding GLDv2 stuff

* Small updates to DELF README

* Clarify that library must be installed before reproducing results

* Merged commit includes the following changes:
319114828  by Andre Araujo:

    Upgrade global feature model exporting to TF2.

--

PiperOrigin-RevId: 319114828

* Properly merging README

* small edits to README

* small edits to README

* small edits to README

* global feature exporting in training README

* Update to DELF README, install instructions

* Centralizing installation instructions

* Small readme update

* Fixing commas

research/delf/INSTALL_INSTRUCTIONS.md

 ## DELF installation
 
+### Installation script
+
+We now have a script to do the entire installation in one shot. Navigate to the
+directory `models/research/delf/delf/python/training`, then run:
+
+```bash
+# From models/research/delf/delf/python/training
+bash install_delf.sh
+```
+
+If this works, you are done! If not, see below for detailed instructions for
+installing this codebase and its dependencies.
+
+*Please note that this installation script only works on 64 bits Linux
+architectures due to the `protoc` binary that is automatically downloaded. If
+you wish to install the DELF library on other architectures please update the
+[`install_delf.sh`](delf/python/training/install_delf.sh) script by referencing
+the desired `protoc`
+[binary release](https://github.com/protocolbuffers/protobuf/releases).*
+
+In more detail: the `install_delf.sh` script installs both the DELF library and
+its dependencies in the following sequence:
+
+*   Install TensorFlow 2.2 and TensorFlow 2.2 for GPU.
+*   Install the [TF-Slim](https://github.com/google-research/tf-slim) library
+    from source.
+*   Download [protoc](https://github.com/protocolbuffers/protobuf) and compile
+    the DELF Protocol Buffers.
+*   Install the matplotlib, numpy, scikit-image, scipy and python3-tk Python
+    libraries.
+*   Install the
+    [TensorFlow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection)
+    from the cloned TensorFlow Model Garden repository.
+*   Install the DELF package.
+
 ### Tensorflow
 
-[![TensorFlow 2.1](https://img.shields.io/badge/tensorflow-2.1-brightgreen)](https://github.com/tensorflow/tensorflow/releases/tag/v2.1.0)
+[![TensorFlow 2.2](https://img.shields.io/badge/tensorflow-2.2-brightgreen)](https://github.com/tensorflow/tensorflow/releases/tag/v2.2.0)
 [![Python 3.6](https://img.shields.io/badge/python-3.6-blue.svg)](https://www.python.org/downloads/release/python-360/)
 
 For detailed steps to install Tensorflow, follow the
@@ -11,9 +46,9 @@ typical user can install Tensorflow using one of the following commands:
 
 ```bash
 # For CPU:
-pip3 install 'tensorflow'
+pip3 install 'tensorflow>=2.2.0'
 # For GPU:
-pip3 install 'tensorflow-gpu'
+pip3 install 'tensorflow-gpu>=2.2.0'
 ```
 
 ### TF-Slim

research/delf/README.md

 # Deep Local and Global Image Features
 
-[![TensorFlow 2.1](https://img.shields.io/badge/tensorflow-2.1-brightgreen)](https://github.com/tensorflow/tensorflow/releases/tag/v2.1.0)
+[![TensorFlow 2.2](https://img.shields.io/badge/tensorflow-2.2-brightgreen)](https://github.com/tensorflow/tensorflow/releases/tag/v2.2.0)
 [![Python 3.6](https://img.shields.io/badge/python-3.6-blue.svg)](https://www.python.org/downloads/release/python-360/)
 
 This project presents code for deep local and global image feature methods,
@@ -182,104 +182,55 @@ directories therein, `protos` and `python`.
 
 ### `delf/protos`
 
-This directory contains protobufs:
-
--   `aggregation_config.proto`: protobuf for configuring local feature
-    aggregation.
--   `box.proto`: protobuf for serializing detected boxes.
--   `datum.proto`: general-purpose protobuf for serializing float tensors.
--   `delf_config.proto`: protobuf for configuring DELF/DELG extraction.
--   `feature.proto`: protobuf for serializing DELF features.
+This directory contains protobufs for local feature aggregation
+(`aggregation_config.proto`), serializing detected boxes (`box.proto`),
+serializing float tensors (`datum.proto`), configuring DELF/DELG extraction
+(`delf_config.proto`), serializing local features (`feature.proto`).
 
 ### `delf/python`
 
-This directory contains files for several different purposes:
-
--   `box_io.py`, `datum_io.py`, `feature_io.py` are helper files for reading and
-    writing tensors and features.
--   `delf_v1.py` contains code to create DELF models.
--   `feature_aggregation_extractor.py` contains a module to perform local
-    feature aggregation.
--   `feature_aggregation_similarity.py` contains a module to perform similarity
-    computation for aggregated local features.
--   `feature_extractor.py` contains the code to extract features using DELF.
-    This is particularly useful for extracting features over multiple scales,
-    with keypoint selection based on attention scores, and PCA/whitening
-    post-processing.
-
-The subdirectory `delf/python/examples` contains sample scripts to run DELF
-feature extraction/matching, and object detection:
-
--   `delf_config_example.pbtxt` shows an example instantiation of the DelfConfig
-    proto, used for DELF feature extraction.
--   `detector.py` is a module to construct an object detector function.
--   `extract_boxes.py` enables object detection from a list of images.
--   `extract_features.py` enables DELF extraction from a list of images.
--   `extractor.py` is a module to construct a DELF/DELG local feature extraction
-    function.
--   `match_images.py` supports image matching using DELF features extracted
-    using `extract_features.py`.
+This directory contains files for several different purposes, such as:
+reading/writing tensors/features (`box_io.py`, `datum_io.py`, `feature_io.py`),
+local feature aggregation extraction and similarity computation
+(`feature_aggregation_extractor.py`, `feature_aggregation_similarity.py`) and
+helper functions for image/feature loading/processing (`utils.py`,
+`feature_extractor.py`).
 
-The subdirectory `delf/python/delg` contains sample scripts/configs related to
-the DELG paper:
+The subdirectory `delf/python/examples` contains sample scripts to run DELF/DELG
+feature extraction/matching (`extractor.py`, `extract_features.py`,
+`match_images.py`) and object detection (`detector.py`, `extract_boxes.py`).
+`delf_config_example.pbtxt` shows an example instantiation of the DelfConfig
+proto, used for DELF feature extraction.
 
--   `delg_gld_config.pbtxt` gives the DelfConfig used in DELG paper.
--   `extract_features.py` for local+global feature extraction on Revisited
-    datasets.
--   `perform_retrieval.py` for performing retrieval/evaluating methods on
-    Revisited datasets.
+The subdirectory `delf/python/delg` contains sample scripts/configs related to
+the DELG paper: `extract_features.py` for local+global feature extraction (with
+and example `delg_gld_config.pbtxt`) and `perform_retrieval.py` for performing
+retrieval/scoring.
 
 The subdirectory `delf/python/detect_to_retrieve` contains sample
-scripts/configs related to the Detect-to-Retrieve paper:
-
--   `aggregation_extraction.py` is a library to extract/save feature
-    aggregation.
--   `boxes_and_features_extraction.py` is a library to extract/save boxes and
-    DELF features.
--   `cluster_delf_features.py` for local feature clustering.
--   `dataset.py` for parsing/evaluating results on Revisited Oxford/Paris
-    datasets.
--   `delf_gld_config.pbtxt` gives the DelfConfig used in Detect-to-Retrieve
-    paper.
--   `extract_aggregation.py` for aggregated local feature extraction.
--   `extract_index_boxes_and_features.py` for index image local feature
-    extraction / bounding box detection on Revisited datasets.
--   `extract_query_features.py` for query image local feature extraction on
-    Revisited datasets.
--   `image_reranking.py` is a module to re-rank images with geometric
-    verification.
--   `perform_retrieval.py` for performing retrieval/evaluating methods using
-    aggregated local features on Revisited datasets.
--   `index_aggregation_config.pbtxt`, `query_aggregation_config.pbtxt` give
-    AggregationConfig's for Detect-to-Retrieve experiments.
+scripts/configs related to the Detect-to-Retrieve paper, for feature/box
+extraction/aggregation/clustering (`aggregation_extraction.py`,
+`boxes_and_features_extraction.py`, `cluster_delf_features.py`,
+`extract_aggregation.py`, `extract_index_boxes_and_features.py`,
+`extract_query_features.py`), image retrieval/reranking (`perform_retrieval.py`,
+`image_reranking.py`), along with configs used for feature
+extraction/aggregation (`delf_gld_config.pbtxt`,
+`index_aggregation_config.pbtxt`, `query_aggregation_config.pbtxt`) and
+Revisited Oxford/Paris dataset parsing/evaluation (`dataset.py`).
 
 The subdirectory `delf/python/google_landmarks_dataset` contains sample
-scripts/modules for computing GLD metrics / reproducing results from the GLDv2
-paper:
-
--   `compute_recognition_metrics.py` performs recognition metric computation
-    given input predictions and solution files.
--   `compute_retrieval_metrics.py` performs retrieval metric computation given
-    input predictions and solution files.
--   `dataset_file_io.py` is a module for dataset-related file IO.
--   `metrics.py` is a module for GLD metric computation.
--   `rn101_af_gldv2clean_config.pbtxt` gives the DelfConfig used in the
-    ResNet101-ArcFace (trained on GLDv2-train-clean) baseline used in the GLDv2
-    paper.
+scripts/modules for computing GLD metrics (`metrics.py`,
+`compute_recognition_metrics.py`, `compute_retrieval_metrics.py`), GLD file IO
+(`dataset_file_io.py`) / reproducing results from the GLDv2 paper
+(`rn101_af_gldv2clean_config.pbtxt` and the instructions therein).
 
 The subdirectory `delf/python/training` contains sample scripts/modules for
-performing DELF training:
-
--   `datasets/googlelandmarks.py` is the dataset module used for training.
--   `model/delf_model.py` is the model module used for training.
--   `model/export_model.py` is a script for exporting trained models in the
-    format used by the inference code.
--   `model/export_model_utils.py` is a module with utilities for model
-    exporting.
--   `model/resnet50.py` is a module with a backbone RN50 implementation.
--   `build_image_dataset.py` converts downloaded dataset into TFRecords format
-    for training.
--   `train.py` is the main training script.
+performing model training (`train.py`) based on a ResNet50 DELF model
+(`model/resnet50.py`, `model/delf_model.py`), also presenting relevant model
+exporting scripts and associated utils (`model/export_model.py`,
+`model/export_global_model.py`, `model/export_model_utils.py`) and dataset
+downloading/preprocessing (`download_dataset.sh`, `build_image_dataset.py`,
+`datasets/googlelandmarks.py`).
 
 Besides these, other files in the different subdirectories contain tests for the
 various modules.
@@ -290,6 +241,16 @@ André Araujo (@andrefaraujo)
 
 ## Release history
 
+### Jul, 2020
+
+-   Full TF2 support. Only one minor `compat.v1` usage left. Updated
+    instructions to require TF2.2
+-   Refactored / much improved training code, with very detailed, step-by-step
+    instructions
+
+**Thanks to contributors**: Dan Anghel, Barbara Fusinska and André
+Araujo.
+
 ### May, 2020
 
 -   Codebase is now Python3-first

research/delf/delf/python/training/README.md

@@ -24,34 +24,9 @@ cd models/research/delf/delf/python/training
 
 ## Install the DELF Library
 
-The DELF Python library can be installed by running the
-[`install_delf.sh`](./install_delf.sh) script using the command:
-
-```
-bash install_delf.sh
-```
-
-The script installs both the DELF library and its dependencies in the following
-sequence:
-
-*   Install TensorFlow 2.2 and TensorFlow 2.2 for GPU.
-*   Install the [TF-Slim](https://github.com/google-research/tf-slim) library
-    from source.
-*   Download [protoc](https://github.com/protocolbuffers/protobuf) and compile
-    the DELF Protocol Buffers.
-*   Install the matplotlib, numpy, scikit-image, scipy and python3-tk Python
-    libraries.
-*   Install the
-    [TensorFlow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection)
-    from the cloned TensorFlow Model Garden repository.
-*   Install the DELF package.
-
-*Please note that the current installation only works on 64 bits Linux
-architectures due to the `protoc` binary downloaded by the installation script.
-If you wish to install the DELF library on other architectures please update the
-[`install_delf.sh`](./install_delf.sh) script by referencing the desired
-`protoc`
-[binary release](https://github.com/protocolbuffers/protobuf/releases).*
+To be able to use this code, please follow
+[these instructions](../../../INSTALL_INSTRUCTIONS.md) to properly install the
+DELF library.
 
 ## Download the GLDv2 Training Data
 

research/delf/setup.py

@@ -22,7 +22,7 @@ install_requires = [
     'pandas >= 0.24.2',
     'numpy >= 1.16.1',
     'scipy >= 1.2.2',
-    'tensorflow >= 2.0.0b1',
+    'tensorflow >= 2.2.0',
     'tf_slim >= 1.1',
     'tensorflow_probability >= 0.9.0',
 ]