Merged commit includes the following changes:

187187978  by Zhichao Lu:

    Only updating hyperparameters if they have non-null values.

--
187097690  by Zhichao Lu:

    Rewrite some conditions a bit more clearly.

--
187085190  by Zhichao Lu:

    More informative error message.

--
186935376  by Zhichao Lu:

    Added option to evaluator.evaluate to use custom evaluator objects.

--
186808249  by Zhichao Lu:

    Fix documentation re: number of stages.

--
186775014  by Zhichao Lu:

    Change anchor generator interface to return a list of BoxLists containing anchors for different feature map layers.

--
186729028  by Zhichao Lu:

    Minor fixes to object detection.

--
186723716  by Zhichao Lu:

    Fix tf_example_decoder.py initailization issue.

--
186668505  by Zhichao Lu:

    Remove unused import.

--
186475361  by Zhichao Lu:

    Update the box predictor interface to return list of predictions - one from each feature map - instead of stacking them into one large tensor.

--
186410844  by Zhichao Lu:

    Fix PythonPath Dependencies.

--
186365384  by Zhichao Lu:

    Made some of the functions in exporter public so they can be reused.

--
186341438  by Zhichao Lu:

    Re-introducing check that label-map-path must be a valid (non-empty) string prior to overwriting pipeline config.

--
186036984  by Zhichao Lu:

    Adding default hyperparameters and allowing for overriding them via flags.

--
186026006  by Zhichao Lu:

    Strip `eval_` prefix from name argument give to TPUEstimator.evaluate since it adds the same prefix internally.

--
186016042  by Zhichao Lu:

    Add an option to evaluate models on training data.

--
185944986  by Zhichao Lu:

    let _update_label_map_path go through even if the path is empty

--
185860781  by Zhichao Lu:

    Add random normal initializer option to hyperparams builder.

    Scale the regression losses outside of the box encoder by adjusting huber loss delta and regression loss weight.

--
185846325  by Zhichao Lu:

    Add an option to normalize localization loss by the code size(number of box coordinates) in SSD Meta architecture.

--
185761217  by Zhichao Lu:

    Change multiscale_grid_anchor_generator to return anchors in normalized coordinates by default and add option to configure it.

    In SSD meta architecture, TargetAssigner operates in normalized coordinate space (i.e, groundtruth boxes are in normalized coordinates) hence we need the option to generate anchors in normalized coordinates.

--
185747733  by Zhichao Lu:

    Change the smooth L1 localization implementationt to use tf.losses.huber_loss and expose the delta parameter in the proto.

--
185715309  by Zhichao Lu:

    Obviates the need for prepadding on mobilenet v1 and v2 for fully convolutional models.

--
185685695  by Zhichao Lu:

    Fix manual stepping schedule to return first rate when there are no boundaries

--
185621650  by Zhichao Lu:

    Added target assigner proto for configuring negative class weights.

--

PiperOrigin-RevId: 187187978

research/object_detection/g3doc/installation.md

@@ -5,7 +5,6 @@
 Tensorflow Object Detection API depends on the following libraries:
 
 *   Protobuf 2.6
-* Python-tk
 *   Pillow 1.0
 *   lxml
 *   tf Slim (which is included in the "tensorflow/models/research/" checkout)
@@ -28,7 +27,7 @@ pip install tensorflow-gpu
 The remaining libraries can be installed on Ubuntu 16.04 using via apt-get:
 
 ``` bash
-sudo apt-get install protobuf-compiler python-pil python-lxml python-tk
+sudo apt-get install protobuf-compiler python-pil python-lxml
 sudo pip install jupyter
 sudo pip install matplotlib
 ```

research/object_detection/g3doc/using_your_own_dataset.md

@@ -103,7 +103,7 @@ FLAGS = flags.FLAGS
 
 
 def create_tf_example(example):
-  # TODO: Populate the following variables from your example.
+  # TODO(user): Populate the following variables from your example.
   height = None # Image height
   width = None # Image width
   filename = None # Filename of the image. Empty if image is not from file
@@ -139,7 +139,7 @@ def create_tf_example(example):
 def main(_):
   writer = tf.python_io.TFRecordWriter(FLAGS.output_path)
 
-  # TODO: Write code to read in your dataset to examples variable
+  # TODO(user): Write code to read in your dataset to examples variable
 
   for example in examples:
     tf_example = create_tf_example(example)

research/object_detection/inference/BUILD

-# Tensorflow Object Detection API: main runnables.
-
-package(
-    default_visibility = ["//visibility:public"],
-)
-
-licenses(["notice"])
-
-# Apache 2.0
-
-py_library(
-    name = "detection_inference",
-    srcs = ["detection_inference.py"],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/core:standard_fields",
-    ],
-)
-
-py_test(
-    name = "detection_inference_test",
-    srcs = ["detection_inference_test.py"],
-    deps = [
-        ":detection_inference",
-        "//PIL:pil",
-        "//numpy",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/core:standard_fields",
-        "//tensorflow/models/research/object_detection/utils:dataset_util",
-    ],
-)
-
-py_binary(
-    name = "infer_detections",
-    srcs = ["infer_detections.py"],
-    deps = [
-        ":detection_inference",
-        "//tensorflow",
-    ],
-)

research/object_detection/inference/detection_inference_test.py

@@ -17,7 +17,6 @@ r"""Tests for detection_inference.py."""
 import os
 import StringIO
 
-
 import numpy as np
 from PIL import Image
 import tensorflow as tf

research/object_detection/inputs_test.py

@@ -34,7 +34,7 @@ FLAGS = tf.flags.FLAGS
 
 def _get_configs_for_model(model_name):
   """Returns configurations for model."""
-  # TODO: Make sure these tests work fine outside google3.
+  # TODO(ronnyvotel): Make sure these tests work fine outside google3.
   fname = os.path.join(
       FLAGS.test_srcdir,
       ('google3/third_party/tensorflow_models/'

research/object_detection/matchers/BUILD

-# Tensorflow Object Detection API: Matcher implementations.
-
-package(
-    default_visibility = ["//visibility:public"],
-)
-
-licenses(["notice"])
-
-# Apache 2.0
-py_library(
-    name = "argmax_matcher",
-    srcs = [
-        "argmax_matcher.py",
-    ],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/core:matcher",
-        "//tensorflow/models/research/object_detection/utils:shape_utils",
-    ],
-)
-
-py_test(
-    name = "argmax_matcher_test",
-    srcs = ["argmax_matcher_test.py"],
-    deps = [
-        ":argmax_matcher",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/utils:test_case",
-    ],
-)
-
-py_library(
-    name = "bipartite_matcher",
-    srcs = [
-        "bipartite_matcher.py",
-    ],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/contrib/image:image_py",
-        "//tensorflow/models/research/object_detection/core:matcher",
-    ],
-)
-
-py_test(
-    name = "bipartite_matcher_test",
-    srcs = [
-        "bipartite_matcher_test.py",
-    ],
-    deps = [
-        ":bipartite_matcher",
-        "//tensorflow",
-    ],
-)

research/object_detection/matchers/bipartite_matcher.py

@@ -38,7 +38,7 @@ class GreedyBipartiteMatcher(matcher.Matcher):
   def _match(self, similarity_matrix, num_valid_rows=-1):
     """Bipartite matches a collection rows and columns. A greedy bi-partite.
 
-    TODO: Add num_valid_columns options to match only that many columns
+    TODO(rathodv): Add num_valid_columns options to match only that many columns
     with all the rows.
 
     Args:

research/object_detection/meta_architectures/BUILD

-# Tensorflow Object Detection API: Meta-architectures.
-
-package(
-    default_visibility = ["//visibility:public"],
-)
-
-licenses(["notice"])
-
-# Apache 2.0
-
-py_library(
-    name = "ssd_meta_arch",
-    srcs = ["ssd_meta_arch.py"],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/core:box_list",
-        "//tensorflow/models/research/object_detection/core:box_predictor",
-        "//tensorflow/models/research/object_detection/core:model",
-        "//tensorflow/models/research/object_detection/core:target_assigner",
-        "//tensorflow/models/research/object_detection/utils:ops",
-        "//tensorflow/models/research/object_detection/utils:shape_utils",
-        "//tensorflow/models/research/object_detection/utils:test_case",
-        "//tensorflow/models/research/object_detection/utils:visualization_utils",
-    ],
-)
-
-py_test(
-    name = "ssd_meta_arch_test",
-    srcs = ["ssd_meta_arch_test.py"],
-    deps = [
-        ":ssd_meta_arch",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/core:anchor_generator",
-        "//tensorflow/models/research/object_detection/core:box_list",
-        "//tensorflow/models/research/object_detection/core:losses",
-        "//tensorflow/models/research/object_detection/core:post_processing",
-        "//tensorflow/models/research/object_detection/core:region_similarity_calculator",
-        "//tensorflow/models/research/object_detection/utils:test_utils",
-    ],
-)
-
-py_library(
-    name = "faster_rcnn_meta_arch",
-    srcs = [
-        "faster_rcnn_meta_arch.py",
-    ],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/anchor_generators:grid_anchor_generator",
-        "//tensorflow/models/research/object_detection/core:balanced_positive_negative_sampler",
-        "//tensorflow/models/research/object_detection/core:box_list",
-        "//tensorflow/models/research/object_detection/core:box_list_ops",
-        "//tensorflow/models/research/object_detection/core:box_predictor",
-        "//tensorflow/models/research/object_detection/core:losses",
-        "//tensorflow/models/research/object_detection/core:model",
-        "//tensorflow/models/research/object_detection/core:post_processing",
-        "//tensorflow/models/research/object_detection/core:standard_fields",
-        "//tensorflow/models/research/object_detection/core:target_assigner",
-        "//tensorflow/models/research/object_detection/utils:ops",
-        "//tensorflow/models/research/object_detection/utils:shape_utils",
-    ],
-)
-
-py_library(
-    name = "faster_rcnn_meta_arch_test_lib",
-    srcs = [
-        "faster_rcnn_meta_arch_test_lib.py",
-    ],
-    deps = [
-        ":faster_rcnn_meta_arch",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/anchor_generators:grid_anchor_generator",
-        "//tensorflow/models/research/object_detection/builders:box_predictor_builder",
-        "//tensorflow/models/research/object_detection/builders:hyperparams_builder",
-        "//tensorflow/models/research/object_detection/builders:post_processing_builder",
-        "//tensorflow/models/research/object_detection/core:losses",
-        "//tensorflow/models/research/object_detection/protos:box_predictor_py_pb2",
-        "//tensorflow/models/research/object_detection/protos:hyperparams_py_pb2",
-        "//tensorflow/models/research/object_detection/protos:post_processing_py_pb2",
-    ],
-)
-
-py_test(
-    name = "faster_rcnn_meta_arch_test",
-    srcs = ["faster_rcnn_meta_arch_test.py"],
-    deps = [
-        ":faster_rcnn_meta_arch_test_lib",
-    ],
-)
-
-py_library(
-    name = "rfcn_meta_arch",
-    srcs = ["rfcn_meta_arch.py"],
-    deps = [
-        ":faster_rcnn_meta_arch",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/core:box_predictor",
-        "//tensorflow/models/research/object_detection/utils:ops",
-    ],
-)
-
-py_test(
-    name = "rfcn_meta_arch_test",
-    srcs = ["rfcn_meta_arch_test.py"],
-    deps = [
-        ":faster_rcnn_meta_arch_test_lib",
-        ":rfcn_meta_arch",
-        "//tensorflow",
-    ],
-)

research/object_detection/meta_architectures/faster_rcnn_meta_arch.py

@@ -365,7 +365,7 @@ class FasterRCNNMetaArch(model.DetectionModel):
       ValueError: If first_stage_anchor_generator is not of type
         grid_anchor_generator.GridAnchorGenerator.
     """
-    # TODO: add_summaries is currently unused. Respect that directive
+    # TODO(rathodv): add_summaries is currently unused. Respect that directive
     # in the future.
     super(FasterRCNNMetaArch, self).__init__(num_classes=num_classes)
 
@@ -597,7 +597,7 @@ class FasterRCNNMetaArch(model.DetectionModel):
           `num_anchors` can differ depending on whether the model is created in
           training or inference mode.
 
-        (and if number_of_stages=1):
+        (and if number_of_stages > 1):
         7) refined_box_encodings: a 3-D tensor with shape
           [total_num_proposals, num_classes, 4] representing predicted
           (final) refined box encodings, where
@@ -910,8 +910,9 @@ class FasterRCNNMetaArch(model.DetectionModel):
         preprocessed_inputs, scope=self.first_stage_feature_extractor_scope)
 
     feature_map_shape = tf.shape(rpn_features_to_crop)
-    anchors = self._first_stage_anchor_generator.generate(
-        [(feature_map_shape[1], feature_map_shape[2])])
+    anchors = box_list_ops.concatenate(
+        self._first_stage_anchor_generator.generate([(feature_map_shape[1],
+                                                      feature_map_shape[2])]))
     with slim.arg_scope(self._first_stage_box_predictor_arg_scope):
       kernel_size = self._first_stage_box_predictor_kernel_size
       rpn_box_predictor_features = slim.conv2d(
@@ -957,9 +958,11 @@ class FasterRCNNMetaArch(model.DetectionModel):
         num_anchors_per_location,
         scope=self.first_stage_box_predictor_scope)
 
-    box_encodings = box_predictions[box_predictor.BOX_ENCODINGS]
-    objectness_predictions_with_background = box_predictions[
-        box_predictor.CLASS_PREDICTIONS_WITH_BACKGROUND]
+    box_encodings = tf.concat(
+        box_predictions[box_predictor.BOX_ENCODINGS], axis=1)
+    objectness_predictions_with_background = tf.concat(
+        box_predictions[box_predictor.CLASS_PREDICTIONS_WITH_BACKGROUND],
+        axis=1)
     return (tf.squeeze(box_encodings, axis=2),
             objectness_predictions_with_background)
 
@@ -1796,7 +1799,7 @@ class FasterRCNNMetaArch(model.DetectionModel):
 
         # Create a new target assigner that matches the proposals to groundtruth
         # and returns the mask targets.
-        # TODO: Move `unmatched_cls_target` from constructor to assign
+        # TODO(rathodv): Move `unmatched_cls_target` from constructor to assign
         # function. This will enable reuse of a single target assigner for both
         # class targets and mask targets.
         mask_target_assigner = target_assigner.create_target_assigner(

research/object_detection/meta_architectures/faster_rcnn_meta_arch_test_lib.py

@@ -745,7 +745,7 @@ class FasterRCNNMetaArchTestBase(tf.test.TestCase):
       preprocessed_inputs, _ = model.preprocess(image_placeholder)
       self.assertAllEqual(preprocessed_inputs.shape.as_list(), image_shape)
 
-  # TODO: Split test into two - with and without masks.
+  # TODO(rathodv): Split test into two - with and without masks.
   def test_loss_first_stage_only_mode(self):
     model = self._build_model(
         is_training=True, number_of_stages=1, second_stage_batch_size=6)
@@ -797,7 +797,7 @@ class FasterRCNNMetaArchTestBase(tf.test.TestCase):
       self.assertTrue('second_stage_localization_loss' not in loss_dict_out)
       self.assertTrue('second_stage_classification_loss' not in loss_dict_out)
 
-  # TODO: Split test into two - with and without masks.
+  # TODO(rathodv): Split test into two - with and without masks.
   def test_loss_full(self):
     model = self._build_model(
         is_training=True, number_of_stages=2, second_stage_batch_size=6)

research/object_detection/meta_architectures/rfcn_meta_arch.py

@@ -164,7 +164,7 @@ class RFCNMetaArch(faster_rcnn_meta_arch.FasterRCNNMetaArch):
       ValueError: If first_stage_anchor_generator is not of type
         grid_anchor_generator.GridAnchorGenerator.
     """
-    # TODO: add_summaries is currently unused. Respect that directive
+    # TODO(rathodv): add_summaries is currently unused. Respect that directive
     # in the future.
     super(RFCNMetaArch, self).__init__(
         is_training,
@@ -275,9 +275,11 @@ class RFCNMetaArch(faster_rcnn_meta_arch.FasterRCNNMetaArch):
         scope=self.second_stage_box_predictor_scope,
         proposal_boxes=proposal_boxes_normalized)
     refined_box_encodings = tf.squeeze(
-        box_predictions[box_predictor.BOX_ENCODINGS], axis=1)
+        tf.concat(box_predictions[box_predictor.BOX_ENCODINGS], axis=1), axis=1)
     class_predictions_with_background = tf.squeeze(
+        tf.concat(
             box_predictions[box_predictor.CLASS_PREDICTIONS_WITH_BACKGROUND],
+            axis=1),
         axis=1)
 
     absolute_proposal_boxes = ops.normalized_to_image_coordinates(

research/object_detection/meta_architectures/ssd_meta_arch.py

@@ -23,6 +23,7 @@ import re
 import tensorflow as tf
 
 from object_detection.core import box_list
+from object_detection.core import box_list_ops
 from object_detection.core import model
 from object_detection.core import standard_fields as fields
 from object_detection.core import target_assigner
@@ -122,6 +123,7 @@ class SSDMetaArch(model.DetectionModel):
                matcher,
                region_similarity_calculator,
                encode_background_as_zeros,
+               negative_class_weight,
                image_resizer_fn,
                non_max_suppression_fn,
                score_conversion_fn,
@@ -131,7 +133,8 @@ class SSDMetaArch(model.DetectionModel):
                localization_loss_weight,
                normalize_loss_by_num_matches,
                hard_example_miner,
-               add_summaries=True):
+               add_summaries=True,
+               normalize_loc_loss_by_codesize=False):
     """SSDMetaArch Constructor.
 
     TODO(rathodv,jonathanhuang): group NMS parameters + score converter into
@@ -151,6 +154,7 @@ class SSDMetaArch(model.DetectionModel):
       encode_background_as_zeros: boolean determining whether background
         targets are to be encoded as an all zeros vector or a one-hot
         vector (where background is the 0th class).
+      negative_class_weight: Weight for confidence loss of negative anchors.
       image_resizer_fn: a callable for image resizing.  This callable always
         takes a rank-3 image tensor (corresponding to a single image) and
         returns a rank-3 image tensor, possibly with new spatial dimensions and
@@ -175,6 +179,8 @@ class SSDMetaArch(model.DetectionModel):
       hard_example_miner: a losses.HardExampleMiner object (can be None)
       add_summaries: boolean (default: True) controlling whether summary ops
         should be added to tensorflow graph.
+      normalize_loc_loss_by_codesize: whether to normalize localization loss
+        by code size of the box encoder.
     """
     super(SSDMetaArch, self).__init__(num_classes=box_predictor.num_classes)
     self._is_training = is_training
@@ -191,7 +197,7 @@ class SSDMetaArch(model.DetectionModel):
     self._matcher = matcher
     self._region_similarity_calculator = region_similarity_calculator
 
-    # TODO: handle agnostic mode and positive/negative class
+    # TODO(jonathanhuang): handle agnostic mode
     # weights
     unmatched_cls_target = None
     unmatched_cls_target = tf.constant([1] + self.num_classes * [0],
@@ -204,7 +210,7 @@ class SSDMetaArch(model.DetectionModel):
         self._region_similarity_calculator,
         self._matcher,
         self._box_coder,
-        negative_class_weight=1.0,
+        negative_class_weight=negative_class_weight,
         unmatched_cls_target=unmatched_cls_target)
 
     self._classification_loss = classification_loss
@@ -212,6 +218,7 @@ class SSDMetaArch(model.DetectionModel):
     self._classification_loss_weight = classification_loss_weight
     self._localization_loss_weight = localization_loss_weight
     self._normalize_loss_by_num_matches = normalize_loss_by_num_matches
+    self._normalize_loc_loss_by_codesize = normalize_loc_loss_by_codesize
     self._hard_example_miner = hard_example_miner
 
     self._image_resizer_fn = image_resizer_fn
@@ -254,7 +261,7 @@ class SSDMetaArch(model.DetectionModel):
     if inputs.dtype is not tf.float32:
       raise ValueError('`preprocess` expects a tf.float32 tensor')
     with tf.name_scope('Preprocessor'):
-      # TODO: revisit whether to always use batch size as
+      # TODO(jonathanhuang): revisit whether to always use batch size as
       # the number of parallel iterations vs allow for dynamic batching.
       outputs = shape_utils.static_or_dynamic_map_fn(
           self._image_resizer_fn,
@@ -344,15 +351,17 @@ class SSDMetaArch(model.DetectionModel):
     feature_map_spatial_dims = self._get_feature_map_spatial_dims(feature_maps)
     image_shape = shape_utils.combined_static_and_dynamic_shape(
         preprocessed_inputs)
-    self._anchors = self._anchor_generator.generate(
+    self._anchors = box_list_ops.concatenate(
+        self._anchor_generator.generate(
             feature_map_spatial_dims,
             im_height=image_shape[1],
-        im_width=image_shape[2])
+            im_width=image_shape[2]))
     prediction_dict = self._box_predictor.predict(
         feature_maps, self._anchor_generator.num_anchors_per_location())
-    box_encodings = tf.squeeze(prediction_dict['box_encodings'], axis=2)
-    class_predictions_with_background = prediction_dict[
-        'class_predictions_with_background']
+    box_encodings = tf.squeeze(
+        tf.concat(prediction_dict['box_encodings'], axis=1), axis=2)
+    class_predictions_with_background = tf.concat(
+        prediction_dict['class_predictions_with_background'], axis=1)
     predictions_dict = {
         'preprocessed_inputs': preprocessed_inputs,
         'box_encodings': box_encodings,
@@ -530,8 +539,11 @@ class SSDMetaArch(model.DetectionModel):
                                 1.0)
 
       with tf.name_scope('localization_loss'):
-        localization_loss = ((self._localization_loss_weight / normalizer) *
-                             localization_loss)
+        localization_loss_normalizer = normalizer
+        if self._normalize_loc_loss_by_codesize:
+          localization_loss_normalizer *= self._box_coder.code_size
+        localization_loss = ((self._localization_loss_weight / (
+            localization_loss_normalizer)) * localization_loss)
       with tf.name_scope('classification_loss'):
         classification_loss = ((self._classification_loss_weight / normalizer) *
                                classification_loss)

research/object_detection/meta_architectures/ssd_meta_arch_test.py

@@ -61,12 +61,12 @@ class MockAnchorGenerator2x2(anchor_generator.AnchorGenerator):
     return [1]
 
   def _generate(self, feature_map_shape_list, im_height, im_width):
-    return box_list.BoxList(
+    return [box_list.BoxList(
         tf.constant([[0, 0, .5, .5],
                      [0, .5, .5, 1],
                      [.5, 0, 1, .5],
                      [1., 1., 1.5, 1.5]  # Anchor that is outside clip_window.
-                    ], tf.float32))
+                    ], tf.float32))]
 
   def num_anchors(self):
     return 4
@@ -74,7 +74,8 @@ class MockAnchorGenerator2x2(anchor_generator.AnchorGenerator):
 
 class SsdMetaArchTest(test_case.TestCase):
 
-  def _create_model(self, apply_hard_mining=True):
+  def _create_model(self, apply_hard_mining=True,
+                    normalize_loc_loss_by_codesize=False):
     is_training = False
     num_classes = 1
     mock_anchor_generator = MockAnchorGenerator2x2()
@@ -98,6 +99,7 @@ class SsdMetaArchTest(test_case.TestCase):
         max_total_size=5)
     classification_loss_weight = 1.0
     localization_loss_weight = 1.0
+    negative_class_weight = 1.0
     normalize_loss_by_num_matches = False
 
     hard_example_miner = None
@@ -111,10 +113,11 @@ class SsdMetaArchTest(test_case.TestCase):
     model = ssd_meta_arch.SSDMetaArch(
         is_training, mock_anchor_generator, mock_box_predictor, mock_box_coder,
         fake_feature_extractor, mock_matcher, region_similarity_calculator,
-        encode_background_as_zeros, image_resizer_fn, non_max_suppression_fn,
-        tf.identity, classification_loss, localization_loss,
-        classification_loss_weight, localization_loss_weight,
-        normalize_loss_by_num_matches, hard_example_miner, add_summaries=False)
+        encode_background_as_zeros, negative_class_weight, image_resizer_fn,
+        non_max_suppression_fn, tf.identity, classification_loss,
+        localization_loss, classification_loss_weight, localization_loss_weight,
+        normalize_loss_by_num_matches, hard_example_miner, add_summaries=False,
+        normalize_loc_loss_by_codesize=normalize_loc_loss_by_codesize)
     return model, num_classes, mock_anchor_generator.num_anchors(), code_size
 
   def test_preprocess_preserves_shapes_with_dynamic_input_image(self):
@@ -287,6 +290,37 @@ class SsdMetaArchTest(test_case.TestCase):
     self.assertAllClose(localization_loss, expected_localization_loss)
     self.assertAllClose(classification_loss, expected_classification_loss)
 
+  def test_loss_results_are_correct_with_normalize_by_codesize_true(self):
+
+    with tf.Graph().as_default():
+      _, _, _, _ = self._create_model()
+    def graph_fn(preprocessed_tensor, groundtruth_boxes1, groundtruth_boxes2,
+                 groundtruth_classes1, groundtruth_classes2):
+      groundtruth_boxes_list = [groundtruth_boxes1, groundtruth_boxes2]
+      groundtruth_classes_list = [groundtruth_classes1, groundtruth_classes2]
+      model, _, _, _ = self._create_model(apply_hard_mining=False,
+                                          normalize_loc_loss_by_codesize=True)
+      model.provide_groundtruth(groundtruth_boxes_list,
+                                groundtruth_classes_list)
+      prediction_dict = model.predict(preprocessed_tensor,
+                                      true_image_shapes=None)
+      loss_dict = model.loss(prediction_dict, true_image_shapes=None)
+      return (loss_dict['localization_loss'],)
+
+    batch_size = 2
+    preprocessed_input = np.random.rand(batch_size, 2, 2, 3).astype(np.float32)
+    groundtruth_boxes1 = np.array([[0, 0, 1, 1]], dtype=np.float32)
+    groundtruth_boxes2 = np.array([[0, 0, 1, 1]], dtype=np.float32)
+    groundtruth_classes1 = np.array([[1]], dtype=np.float32)
+    groundtruth_classes2 = np.array([[1]], dtype=np.float32)
+    expected_localization_loss = 0.5 / 4
+    localization_loss = self.execute(graph_fn, [preprocessed_input,
+                                                groundtruth_boxes1,
+                                                groundtruth_boxes2,
+                                                groundtruth_classes1,
+                                                groundtruth_classes2])
+    self.assertAllClose(localization_loss, expected_localization_loss)
+
   def test_loss_results_are_correct_with_hard_example_mining(self):
 
     with tf.Graph().as_default():

research/object_detection/metrics/BUILD

-# Tensorflow Object Detection API: main runnables.
-
-package(
-    default_visibility = ["//visibility:public"],
-)
-
-licenses(["notice"])
-
-# Apache 2.0
-
-py_library(
-    name = "coco_tools",
-    srcs = [
-        "coco_tools.py",
-    ],
-    deps = [
-        "//file/localfile",
-        "//file/placer",
-        "//pycocotools",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/utils:json_utils",
-    ],
-)
-
-py_test(
-    name = "coco_tools_test",
-    srcs = [
-        "coco_tools_test.py",
-    ],
-    deps = [
-        ":coco_tools",
-        "//testing/pybase",
-        "//numpy",
-    ],
-)
-
-py_library(
-    name = "coco_evaluation",
-    srcs = [
-        "coco_evaluation.py",
-    ],
-    deps = [
-        ":coco_tools",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/core:standard_fields",
-        "//tensorflow/models/research/object_detection/utils:object_detection_evaluation",
-    ],
-)
-
-py_test(
-    name = "coco_evaluation_test",
-    srcs = [
-        "coco_evaluation_test.py",
-    ],
-    deps = [
-        ":coco_evaluation",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/core:standard_fields",
-    ],
-)
-
-py_binary(
-    name = "offline_eval_map_corloc",
-    srcs = [
-        "offline_eval_map_corloc.py",
-    ],
-    deps = [
-        ":tf_example_parser",
-        "//tensorflow/models/research/object_detection:evaluator",
-        "//tensorflow/models/research/object_detection/builders:input_reader_builder",
-        "//tensorflow/models/research/object_detection/core:standard_fields",
-        "//tensorflow/models/research/object_detection/utils:config_util",
-        "//tensorflow/models/research/object_detection/utils:label_map_util",
-    ],
-)
-
-py_test(
-    name = "offline_eval_map_corloc_test",
-    srcs = [
-        "offline_eval_map_corloc_test.py",
-    ],
-    deps = [
-        ":offline_eval_map_corloc",
-        "//tensorflow",
-    ],
-)
-
-py_library(
-    name = "tf_example_parser",
-    srcs = ["tf_example_parser.py"],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/core:data_parser",
-        "//tensorflow/models/research/object_detection/core:standard_fields",
-    ],
-)
-
-py_test(
-    name = "tf_example_parser_test",
-    srcs = ["tf_example_parser_test.py"],
-    deps = [
-        ":tf_example_parser",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/core:standard_fields",
-    ],
-)

research/object_detection/metrics/coco_tools.py

@@ -17,7 +17,7 @@
 Note that nothing in this file is tensorflow related and thus cannot
 be called directly as a slim metric, for example.
 
-TODO: wrap as a slim metric in metrics.py
+TODO(jonathanhuang): wrap as a slim metric in metrics.py
 
 
 Usage example: given a set of images with ids in the list image_ids
@@ -339,7 +339,7 @@ def ExportSingleImageGroundtruthToCoco(image_id,
 
   In the exported result, "area" fields are always set to the area of the
   groundtruth bounding box and "iscrowd" fields are always set to 0.
-  TODO: pass in "iscrowd" array for evaluating on COCO dataset.
+  TODO(jonathanhuang): pass in "iscrowd" array for evaluating on COCO dataset.
 
   Args:
     image_id: a unique image identifier either of type integer or string.
@@ -416,7 +416,7 @@ def ExportGroundtruthToCOCO(image_ids,
 
   In the exported result, "area" fields are always set to the area of the
   groundtruth bounding box and "iscrowd" fields are always set to 0.
-  TODO: pass in "iscrowd" array for evaluating on COCO dataset.
+  TODO(jonathanhuang): pass in "iscrowd" array for evaluating on COCO dataset.
 
   Args:
     image_ids: a list of unique image identifier either of type integer or

research/object_detection/model.py

@@ -319,9 +319,11 @@ def create_model_fn(detection_model_fn, configs, hparams, use_tpu=False):
       else:
         category_index = label_map_util.create_category_index_from_labelmap(
             eval_input_config.label_map_path)
-      detection_and_groundtruth = vis_utils.draw_side_by_side_evaluation_image(
-          eval_dict, category_index, max_boxes_to_draw=20, min_score_thresh=0.2)
       if not use_tpu:
+        detection_and_groundtruth = (
+            vis_utils.draw_side_by_side_evaluation_image(
+                eval_dict, category_index, max_boxes_to_draw=20,
+                min_score_thresh=0.2))
         tf.summary.image('Detections_Left_Groundtruth_Right',
                          detection_and_groundtruth)
 
@@ -424,11 +426,11 @@ def populate_experiment(run_config,
   eval_config = configs['eval_config']
   eval_input_config = configs['eval_input_config']
 
-  if train_steps is None:
-    train_steps = train_config.num_steps if train_config.num_steps else None
+  if train_steps is None and train_config.num_steps:
+    train_steps = train_config.num_steps
 
-  if eval_steps is None:
-    eval_steps = eval_config.num_examples if eval_config.num_examples else None
+  if eval_steps is None and eval_config.num_examples:
+    eval_steps = eval_config.num_examples
 
   detection_model_fn = functools.partial(
       model_builder.build, model_config=model_config)

research/object_detection/model_tpu.py

@@ -77,6 +77,10 @@ tf.flags.DEFINE_integer('min_eval_interval_secs', 180,
 tf.flags.DEFINE_integer(
     'eval_timeout_secs', None,
     'Maximum seconds between checkpoints before evaluation terminates.')
+tf.flags.DEFINE_string('hparams_overrides', None, 'Comma-separated list of '
+                       'hyperparameters to override defaults.')
+tf.flags.DEFINE_boolean('eval_training_data', False,
+                        'If training data should be evaluated for this job.')
 
 FLAGS = tf.flags.FLAGS
 
@@ -122,7 +126,10 @@ def create_estimator(run_config,
   Returns:
     Estimator: A estimator object used for training and evaluation
     train_input_fn: Input function for the training loop
-    eval_input_fn: Input function for the evaluation run
+    eval_validation_input_fn: Input function to run for evaluation on
+      validation data.
+    eval_training_input_fn: Input function to run for evaluation on
+      training data.
     train_steps: Number of training steps either from arg `train_steps` or
       `TrainConfig` proto
     eval_steps: Number of evaluation steps either from arg `eval_steps` or
@@ -141,15 +148,17 @@ def create_estimator(run_config,
   train_input_config = configs['train_input_config']
   eval_config = configs['eval_config']
   eval_input_config = configs['eval_input_config']
+  if FLAGS.eval_training_data:
+    eval_input_config = configs['train_input_config']
 
   if params is None:
     params = {}
 
-  if train_steps is None:
-    train_steps = train_config.num_steps if train_config.num_steps else None
+  if train_steps is None and train_config.num_steps:
+    train_steps = train_config.num_steps
 
-  if eval_steps is None:
-    eval_steps = eval_config.num_examples if eval_config.num_examples else None
+  if eval_steps is None and eval_config.num_examples:
+    eval_steps = eval_config.num_examples
 
   detection_model_fn = functools.partial(
       model_builder.build, model_config=model_config)
@@ -159,10 +168,14 @@ def create_estimator(run_config,
       train_config=train_config,
       train_input_config=train_input_config,
       model_config=model_config)
-  eval_input_fn = inputs.create_eval_input_fn(
+  eval_validation_input_fn = inputs.create_eval_input_fn(
       eval_config=eval_config,
       eval_input_config=eval_input_config,
       model_config=model_config)
+  eval_training_input_fn = inputs.create_eval_input_fn(
+      eval_config=eval_config,
+      eval_input_config=train_input_config,
+      model_config=model_config)
 
   estimator = tpu_estimator.TPUEstimator(
       model_fn=model_fn_creator(detection_model_fn, configs, hparams,
@@ -173,7 +186,8 @@ def create_estimator(run_config,
       use_tpu=use_tpu,
       config=run_config,
       params=params)
-  return estimator, train_input_fn, eval_input_fn, train_steps, eval_steps
+  return (estimator, train_input_fn, eval_validation_input_fn,
+          eval_training_input_fn, train_steps, eval_steps)
 
 
 def main(unused_argv):
@@ -204,10 +218,12 @@ def main(unused_argv):
           iterations_per_loop=FLAGS.iterations_per_loop,
           num_shards=FLAGS.num_shards))
   params = {}
-  estimator, train_input_fn, eval_input_fn, train_steps, eval_steps = (
+  (estimator, train_input_fn, eval_validation_input_fn, eval_training_input_fn,
+   train_steps, eval_steps) = (
        create_estimator(
            config,
-          model_hparams.create_hparams(),
+           model_hparams.create_hparams(
+               hparams_overrides=FLAGS.hparams_overrides),
            FLAGS.pipeline_config_path,
            train_steps=FLAGS.num_train_steps,
            eval_steps=FLAGS.num_eval_steps,
@@ -221,7 +237,8 @@ def main(unused_argv):
 
   if FLAGS.mode == 'train_and_eval':
     # Eval one time.
-    eval_results = estimator.evaluate(input_fn=eval_input_fn, steps=eval_steps)
+    eval_results = estimator.evaluate(
+        input_fn=eval_validation_input_fn, steps=eval_steps)
     tf.logging.info('Eval results: %s' % eval_results)
 
   # Continuously evaluating.
@@ -239,11 +256,18 @@ def main(unused_argv):
         timeout_fn=terminate_eval):
 
       tf.logging.info('Starting to evaluate.')
+      if FLAGS.eval_training_data:
+        name = 'training_data'
+        input_fn = eval_training_input_fn
+      else:
+        name = 'validation_data'
+        input_fn = eval_validation_input_fn
       try:
         eval_results = estimator.evaluate(
-            input_fn=eval_input_fn,
+            input_fn=input_fn,
             steps=eval_steps,
-            checkpoint_path=ckpt)
+            checkpoint_path=ckpt,
+            name=name)
         tf.logging.info('Eval results: %s' % eval_results)
 
         # Terminate eval job when final checkpoint is reached

research/object_detection/models/BUILD

-# Tensorflow Object Detection API: Models.
-
-package(
-    default_visibility = ["//visibility:public"],
-)
-
-licenses(["notice"])
-
-# Apache 2.0
-
-py_library(
-    name = "feature_map_generators",
-    srcs = [
-        "feature_map_generators.py",
-    ],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/utils:ops",
-    ],
-)
-
-py_test(
-    name = "feature_map_generators_test",
-    srcs = [
-        "feature_map_generators_test.py",
-    ],
-    deps = [
-        ":feature_map_generators",
-        "//tensorflow",
-    ],
-)
-
-py_library(
-    name = "ssd_feature_extractor_test",
-    srcs = [
-        "ssd_feature_extractor_test.py",
-    ],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/utils:test_case",
-    ],
-)
-
-py_library(
-    name = "ssd_inception_v2_feature_extractor",
-    srcs = [
-        "ssd_inception_v2_feature_extractor.py",
-    ],
-    deps = [
-        ":feature_map_generators",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/meta_architectures:ssd_meta_arch",
-        "//tensorflow/models/research/object_detection/utils:ops",
-        "//tensorflow/models/research/object_detection/utils:shape_utils",
-        "//third_party/tensorflow_models/slim:inception_v2",
-    ],
-)
-
-py_library(
-    name = "ssd_inception_v3_feature_extractor",
-    srcs = [
-        "ssd_inception_v3_feature_extractor.py",
-    ],
-    deps = [
-        ":feature_map_generators",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/meta_architectures:ssd_meta_arch",
-        "//tensorflow/models/research/object_detection/utils:ops",
-        "//tensorflow/models/research/object_detection/utils:shape_utils",
-        "//third_party/tensorflow_models/slim:inception_v3",
-    ],
-)
-
-py_library(
-    name = "ssd_mobilenet_v1_feature_extractor",
-    srcs = ["ssd_mobilenet_v1_feature_extractor.py"],
-    deps = [
-        ":feature_map_generators",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/meta_architectures:ssd_meta_arch",
-        "//tensorflow/models/research/object_detection/utils:ops",
-        "//tensorflow/models/research/object_detection/utils:shape_utils",
-        "//third_party/tensorflow_models/slim:mobilenet_v1",
-    ],
-)
-
-py_library(
-    name = "embedded_ssd_mobilenet_v1_feature_extractor",
-    srcs = ["embedded_ssd_mobilenet_v1_feature_extractor.py"],
-    deps = [
-        ":feature_map_generators",
-        ":ssd_mobilenet_v1_feature_extractor",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/utils:ops",
-        "//third_party/tensorflow_models/slim:mobilenet_v1",
-    ],
-)
-
-py_library(
-    name = "ssd_resnet_v1_fpn_feature_extractor",
-    srcs = ["ssd_resnet_v1_fpn_feature_extractor.py"],
-    deps = [
-        ":feature_map_generators",
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/meta_architectures:ssd_meta_arch",
-        "//tensorflow/models/research/object_detection/utils:ops",
-        "//tensorflow/models/research/object_detection/utils:shape_utils",
-        "//third_party/tensorflow_models/slim:resnet_v1",
-    ],
-)
-
-py_library(
-    name = "ssd_resnet_v1_fpn_feature_extractor_testbase",
-    srcs = ["ssd_resnet_v1_fpn_feature_extractor_testbase.py"],
-    deps = [
-        "//tensorflow/models/research/object_detection/models:ssd_feature_extractor_test",
-    ],
-)
-
-py_test(
-    name = "ssd_resnet_v1_fpn_feature_extractor_test",
-    timeout = "long",
-    srcs = ["ssd_resnet_v1_fpn_feature_extractor_test.py"],
-    deps = [
-        ":ssd_resnet_v1_fpn_feature_extractor",
-        ":ssd_resnet_v1_fpn_feature_extractor_testbase",
-        "//tensorflow",
-    ],
-)
-
-py_test(
-    name = "ssd_inception_v2_feature_extractor_test",
-    srcs = [
-        "ssd_inception_v2_feature_extractor_test.py",
-    ],
-    deps = [
-        ":ssd_feature_extractor_test",
-        ":ssd_inception_v2_feature_extractor",
-        "//tensorflow",
-    ],
-)
-
-py_test(
-    name = "ssd_inception_v3_feature_extractor_test",
-    srcs = [
-        "ssd_inception_v3_feature_extractor_test.py",
-    ],
-    deps = [
-        ":ssd_feature_extractor_test",
-        ":ssd_inception_v3_feature_extractor",
-        "//tensorflow",
-    ],
-)
-
-py_test(
-    name = "ssd_mobilenet_v1_feature_extractor_test",
-    srcs = ["ssd_mobilenet_v1_feature_extractor_test.py"],
-    deps = [
-        ":ssd_feature_extractor_test",
-        ":ssd_mobilenet_v1_feature_extractor",
-        "//tensorflow",
-    ],
-)
-
-py_test(
-    name = "embedded_ssd_mobilenet_v1_feature_extractor_test",
-    srcs = ["embedded_ssd_mobilenet_v1_feature_extractor_test.py"],
-    deps = [
-        ":embedded_ssd_mobilenet_v1_feature_extractor",
-        ":ssd_feature_extractor_test",
-        "//tensorflow",
-    ],
-)
-
-py_test(
-    name = "faster_rcnn_nas_feature_extractor_test",
-    srcs = [
-        "faster_rcnn_nas_feature_extractor_test.py",
-    ],
-    deps = [
-        ":faster_rcnn_nas_feature_extractor",
-        "//tensorflow",
-    ],
-)
-
-py_library(
-    name = "faster_rcnn_nas_feature_extractor",
-    srcs = [
-        "faster_rcnn_nas_feature_extractor.py",
-    ],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/meta_architectures:faster_rcnn_meta_arch",
-        "//third_party/tensorflow_models/slim:nasnet",
-    ],
-)
-
-py_library(
-    name = "faster_rcnn_inception_resnet_v2_feature_extractor",
-    srcs = [
-        "faster_rcnn_inception_resnet_v2_feature_extractor.py",
-    ],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/meta_architectures:faster_rcnn_meta_arch",
-        "//third_party/tensorflow_models/slim:inception_resnet_v2",
-    ],
-)
-
-py_test(
-    name = "faster_rcnn_inception_resnet_v2_feature_extractor_test",
-    srcs = [
-        "faster_rcnn_inception_resnet_v2_feature_extractor_test.py",
-    ],
-    deps = [
-        ":faster_rcnn_inception_resnet_v2_feature_extractor",
-        "//tensorflow",
-    ],
-)
-
-py_library(
-    name = "faster_rcnn_inception_v2_feature_extractor",
-    srcs = [
-        "faster_rcnn_inception_v2_feature_extractor.py",
-    ],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/meta_architectures:faster_rcnn_meta_arch",
-        "//third_party/tensorflow_models/slim:inception_v2",
-    ],
-)
-
-py_test(
-    name = "faster_rcnn_inception_v2_feature_extractor_test",
-    srcs = [
-        "faster_rcnn_inception_v2_feature_extractor_test.py",
-    ],
-    deps = [
-        ":faster_rcnn_inception_v2_feature_extractor",
-        "//tensorflow",
-    ],
-)
-
-py_library(
-    name = "faster_rcnn_resnet_v1_feature_extractor",
-    srcs = [
-        "faster_rcnn_resnet_v1_feature_extractor.py",
-    ],
-    deps = [
-        "//tensorflow",
-        "//tensorflow/models/research/object_detection/meta_architectures:faster_rcnn_meta_arch",
-        "//third_party/tensorflow_models/slim:resnet_utils",
-        "//third_party/tensorflow_models/slim:resnet_v1",
-    ],
-)
-
-py_test(
-    name = "faster_rcnn_resnet_v1_feature_extractor_test",
-    srcs = [
-        "faster_rcnn_resnet_v1_feature_extractor_test.py",
-    ],
-    deps = [
-        ":faster_rcnn_resnet_v1_feature_extractor",
-        "//tensorflow",
-    ],
-)

research/object_detection/models/faster_rcnn_nas_feature_extractor.py

@@ -108,7 +108,7 @@ def _build_nasnet_base(hidden_previous,
   return net
 
 
-# TODO: Only fixed_shape_resizer is currently supported for NASNet
+# TODO(shlens): Only fixed_shape_resizer is currently supported for NASNet
 # featurization. The reason for this is that nasnet.py only supports
 # inputs with fully known shapes. We need to update nasnet.py to handle
 # shapes not known at compile time.

research/object_detection/models/faster_rcnn_resnet_v1_feature_extractor.py

@@ -111,7 +111,7 @@ class FasterRCNNResnetV1FeatureExtractor(
 
     with tf.control_dependencies([shape_assert]):
       # Disables batchnorm for fine-tuning with smaller batch sizes.
-      # TODO: Figure out if it is needed when image
+      # TODO(chensun): Figure out if it is needed when image
       # batch size is bigger.
       with slim.arg_scope(
           resnet_utils.resnet_arg_scope(