Merged commit includes the following changes: (#6932)

250447559  by Zhichao Lu:

    Update expected files format for Instance Segmentation challenge:
    - add fields ImageWidth, ImageHeight and store the values per prediction
    - as mask, store only encoded image and assume its size is ImageWidth x ImageHeight

--
250402780  by rathodv:

    Fix failing Mask R-CNN TPU convergence test.

    Cast second stage prediction tensors from bfloat16 to float32 to prevent errors in third target assignment (Mask Prediction) - Concat with different types bfloat16 and bfloat32 isn't allowed.

--
250300240  by Zhichao Lu:

    Addion Open Images Challenge 2019 object detection and instance segmentation
    support into Estimator framework.

--
249944839  by rathodv:

    Modify exporter.py to add multiclass score nodes in exported inference graphs.

--
249935201  by rathodv:

    Modify postprocess methods to preserve multiclass scores after non max suppression.

--
249878079  by Zhichao Lu:

    This CL slightly refactors some Object Detection helper functions for data creation, evaluation, and groundtruth providing.

    This will allow the eager+function custom loops to share code with the existing estimator training loops.

    Concretely we make the following changes:
    1. In input creation we separate dataset-creation into top-level helpers, and allow it to optionally accept a pre-constructed model directly instead of always creating a model from the config just for feature preprocessing.

    2. In coco evaluation we split the update_op creation into its own function, which the custom loops will call directly.

    3. In model_lib we move groundtruth providing/ datastructure munging into a helper function

    4. For now we put an escape hatch in `_summarize_target_assignment` when executing in tf v2.0 behavior because the summary apis used only work w/ tf 1.x

--
249673507  by rathodv:

    Use explicit casts instead of tf.to_float and tf.to_int32 to avoid warnings.

--
249656006  by Zhichao Lu:

    Add named "raw_keypoint_locations" node that corresponds with the "raw_box_locations" node.

--
249651674  by rathodv:

    Keep proposal boxes in float format. MatMulCropAndResize can handle the type even when feature themselves are bfloat16s.

--
249568633  by rathodv:

    Support q > 1 in class agnostic NMS.
    Break post_processing_test.py into 3 separate files to avoid linter errors.

--
249535530  by rathodv:

    Update some deprecated arguments to tf ops.

--
249368223  by rathodv:

    Modify MatMulCropAndResize to use MultiLevelRoIAlign method and move the tests to spatial_transform_ops.py module.

    This cl establishes that CropAndResize and RoIAlign are equivalent and only differ in the sampling point grid within the boxes. CropAndResize uses a uniform size x size point grid such that the corner points exactly overlap box corners, while RoiAlign divides boxes into size x size cells and uses their centers as sampling points. In this cl, we switch MatMulCropAndResize to use the MultiLevelRoIAlign implementation with `align_corner` option as MultiLevelRoIAlign implementation is more memory efficient on TPU when compared to the original MatMulCropAndResize.

--
249337338  by chowdhery:

    Add class-agnostic non-max-suppression in post_processing

--
249139196  by Zhichao Lu:

    Fix positional argument bug in export_tflite_ssd_graph

--
249120219  by Zhichao Lu:

    Add evaluator for computing precision limited to a given recall range.

--
249030593  by Zhichao Lu:

    Evaluation util to run segmentation and detection challenge evaluation.

--
248554358  by Zhichao Lu:

    This change contains the auxiliary changes required for TF 2.0 style training with eager+functions+dist strat loops, but not the loops themselves.

    It includes:
    - Updates to shape usage to support both tensorshape v1 and tensorshape v2
    - A fix to FreezableBatchNorm to not override the `training` arg in call when `None` was passed to the constructor (Not an issue in the estimator loops but it was in the custom loops)
    - Puts some constants in init_scope so they work in eager + functions
    - Makes learning rate schedules return a callable in eager mode (required so they update when the global_step changes)
    - Makes DetectionModel a tf.module so it tracks variables (e.g. ones nested in layers)
    - Removes some references to `op.name` for some losses and replaces it w/ explicit names
    - A small part of the change to allow the coco evaluation metrics to work in eager mode

--
248271226  by rathodv:

    Add MultiLevel RoIAlign op.

--
248229103  by rathodv:

    Add functions to 1. pad features maps 2. ravel 5-D indices

--
248206769  by rathodv:

    Add utilities needed to introduce RoI Align op.

--
248177733  by pengchong:

    Internal changes

--
247742582  by Zhichao Lu:

    Open Images Challenge 2019 instance segmentation metric: part 2

--
247525401  by Zhichao Lu:

    Update comments on max_class_per_detection.

--
247520753  by rathodv:

    Add multilevel crop and resize operation that builds on top of matmul_crop_and_resize.

--
247391600  by Zhichao Lu:

    Open Images Challenge 2019 instance segmentation metric

--
247325813  by chowdhery:

    Quantized MobileNet v2 SSD FPNLite config with depth multiplier 0.75

--

PiperOrigin-RevId: 250447559

research/object_detection/anchor_generators/flexible_grid_anchor_generator.py

@@ -111,11 +111,11 @@ class FlexibleGridAnchorGenerator(anchor_generator.AnchorGenerator):
       anchor_grid = grid_anchor_generator.tile_anchors(
           feat_shape[0],
           feat_shape[1],
-          tf.to_float(tf.convert_to_tensor(base_sizes)),
-          tf.to_float(tf.convert_to_tensor(aspect_ratios)),
+          tf.cast(tf.convert_to_tensor(base_sizes), dtype=tf.float32),
+          tf.cast(tf.convert_to_tensor(aspect_ratios), dtype=tf.float32),
           tf.constant([1.0, 1.0]),
-          tf.to_float(tf.convert_to_tensor(anchor_stride)),
-          tf.to_float(tf.convert_to_tensor(anchor_offset)))
+          tf.cast(tf.convert_to_tensor(anchor_stride), dtype=tf.float32),
+          tf.cast(tf.convert_to_tensor(anchor_offset), dtype=tf.float32))
       num_anchors = anchor_grid.num_boxes_static()
       if num_anchors is None:
         num_anchors = anchor_grid.num_boxes()

research/object_detection/anchor_generators/grid_anchor_generator.py

@@ -105,12 +105,16 @@ class GridAnchorGenerator(anchor_generator.AnchorGenerator):
     if not all([isinstance(list_item, tuple) and len(list_item) == 2
                 for list_item in feature_map_shape_list]):
       raise ValueError('feature_map_shape_list must be a list of pairs.')
-    self._base_anchor_size = tf.to_float(tf.convert_to_tensor(
-        self._base_anchor_size))
-    self._anchor_stride = tf.to_float(tf.convert_to_tensor(
-        self._anchor_stride))
-    self._anchor_offset = tf.to_float(tf.convert_to_tensor(
-        self._anchor_offset))
+
+    # Create constants in init_scope so they can be created in tf.functions
+    # and accessed from outside of the function.
+    with tf.init_scope():
+      self._base_anchor_size = tf.cast(tf.convert_to_tensor(
+          self._base_anchor_size), dtype=tf.float32)
+      self._anchor_stride = tf.cast(tf.convert_to_tensor(
+          self._anchor_stride), dtype=tf.float32)
+      self._anchor_offset = tf.cast(tf.convert_to_tensor(
+          self._anchor_offset), dtype=tf.float32)
 
     grid_height, grid_width = feature_map_shape_list[0]
     scales_grid, aspect_ratios_grid = ops.meshgrid(self._scales,
@@ -179,9 +183,9 @@ def tile_anchors(grid_height,
   widths = scales * ratio_sqrts * base_anchor_size[1]
 
   # Get a grid of box centers
-  y_centers = tf.to_float(tf.range(grid_height))
+  y_centers = tf.cast(tf.range(grid_height), dtype=tf.float32)
   y_centers = y_centers * anchor_stride[0] + anchor_offset[0]
-  x_centers = tf.to_float(tf.range(grid_width))
+  x_centers = tf.cast(tf.range(grid_width), dtype=tf.float32)
   x_centers = x_centers * anchor_stride[1] + anchor_offset[1]
   x_centers, y_centers = ops.meshgrid(x_centers, y_centers)
 

research/object_detection/anchor_generators/multiple_grid_anchor_generator.py

@@ -180,22 +180,23 @@ class MultipleGridAnchorGenerator(anchor_generator.AnchorGenerator):
                 for list_item in feature_map_shape_list]):
       raise ValueError('feature_map_shape_list must be a list of pairs.')
 
-    im_height = tf.to_float(im_height)
-    im_width = tf.to_float(im_width)
+    im_height = tf.cast(im_height, dtype=tf.float32)
+    im_width = tf.cast(im_width, dtype=tf.float32)
 
     if not self._anchor_strides:
-      anchor_strides = [(1.0 / tf.to_float(pair[0]), 1.0 / tf.to_float(pair[1]))
+      anchor_strides = [(1.0 / tf.cast(pair[0], dtype=tf.float32),
+                         1.0 / tf.cast(pair[1], dtype=tf.float32))
                         for pair in feature_map_shape_list]
     else:
-      anchor_strides = [(tf.to_float(stride[0]) / im_height,
-                         tf.to_float(stride[1]) / im_width)
+      anchor_strides = [(tf.cast(stride[0], dtype=tf.float32) / im_height,
+                         tf.cast(stride[1], dtype=tf.float32) / im_width)
                         for stride in self._anchor_strides]
     if not self._anchor_offsets:
       anchor_offsets = [(0.5 * stride[0], 0.5 * stride[1])
                         for stride in anchor_strides]
     else:
-      anchor_offsets = [(tf.to_float(offset[0]) / im_height,
-                         tf.to_float(offset[1]) / im_width)
+      anchor_offsets = [(tf.cast(offset[0], dtype=tf.float32) / im_height,
+                         tf.cast(offset[1], dtype=tf.float32) / im_width)
                         for offset in self._anchor_offsets]
 
     for arg, arg_name in zip([anchor_strides, anchor_offsets],

research/object_detection/box_coders/keypoint_box_coder.py

@@ -66,9 +66,11 @@ class KeypointBoxCoder(box_coder.BoxCoder):
     self._scale_factors = scale_factors
     self._keypoint_scale_factors = None
     if scale_factors is not None:
-      self._keypoint_scale_factors = tf.expand_dims(tf.tile(
-          [tf.to_float(scale_factors[0]), tf.to_float(scale_factors[1])],
-          [num_keypoints]), 1)
+      self._keypoint_scale_factors = tf.expand_dims(
+          tf.tile([
+              tf.cast(scale_factors[0], dtype=tf.float32),
+              tf.cast(scale_factors[1], dtype=tf.float32)
+          ], [num_keypoints]), 1)
 
   @property
   def code_size(self):

research/object_detection/builders/image_resizer_builder_test.py

@@ -27,8 +27,9 @@ class ImageResizerBuilderTest(tf.test.TestCase):
     image_resizer_config = image_resizer_pb2.ImageResizer()
     text_format.Merge(text_proto, image_resizer_config)
     image_resizer_fn = image_resizer_builder.build(image_resizer_config)
-    images = tf.to_float(
-        tf.random_uniform(input_shape, minval=0, maxval=255, dtype=tf.int32))
+    images = tf.cast(
+        tf.random_uniform(input_shape, minval=0, maxval=255, dtype=tf.int32),
+        dtype=tf.float32)
     resized_images, _ = image_resizer_fn(images)
     with self.test_session() as sess:
       return sess.run(resized_images).shape

research/object_detection/builders/optimizer_builder.py

@@ -21,11 +21,13 @@ import tensorflow as tf
 from object_detection.utils import learning_schedules
 
 
-def build(optimizer_config):
+def build(optimizer_config, global_step=None):
   """Create optimizer based on config.
 
   Args:
     optimizer_config: A Optimizer proto message.
+    global_step: A variable representing the current step.
+      If None, defaults to tf.train.get_or_create_global_step()
 
   Returns:
     An optimizer and a list of variables for summary.
@@ -39,7 +41,8 @@ def build(optimizer_config):
   summary_vars = []
   if optimizer_type == 'rms_prop_optimizer':
     config = optimizer_config.rms_prop_optimizer
-    learning_rate = _create_learning_rate(config.learning_rate)
+    learning_rate = _create_learning_rate(config.learning_rate,
+                                          global_step=global_step)
     summary_vars.append(learning_rate)
     optimizer = tf.train.RMSPropOptimizer(
         learning_rate,
@@ -49,7 +52,8 @@ def build(optimizer_config):
 
   if optimizer_type == 'momentum_optimizer':
     config = optimizer_config.momentum_optimizer
-    learning_rate = _create_learning_rate(config.learning_rate)
+    learning_rate = _create_learning_rate(config.learning_rate,
+                                          global_step=global_step)
     summary_vars.append(learning_rate)
     optimizer = tf.train.MomentumOptimizer(
         learning_rate,
@@ -57,7 +61,8 @@ def build(optimizer_config):
 
   if optimizer_type == 'adam_optimizer':
     config = optimizer_config.adam_optimizer
-    learning_rate = _create_learning_rate(config.learning_rate)
+    learning_rate = _create_learning_rate(config.learning_rate,
+                                          global_step=global_step)
     summary_vars.append(learning_rate)
     optimizer = tf.train.AdamOptimizer(learning_rate)
 
@@ -72,11 +77,13 @@ def build(optimizer_config):
   return optimizer, summary_vars
 
 
-def _create_learning_rate(learning_rate_config):
+def _create_learning_rate(learning_rate_config, global_step=None):
   """Create optimizer learning rate based on config.
 
   Args:
     learning_rate_config: A LearningRate proto message.
+    global_step: A variable representing the current step.
+      If None, defaults to tf.train.get_or_create_global_step()
 
   Returns:
     A learning rate.
@@ -84,6 +91,8 @@ def _create_learning_rate(learning_rate_config):
   Raises:
     ValueError: when using an unsupported input data type.
   """
+  if global_step is None:
+    global_step = tf.train.get_or_create_global_step()
   learning_rate = None
   learning_rate_type = learning_rate_config.WhichOneof('learning_rate')
   if learning_rate_type == 'constant_learning_rate':
@@ -94,7 +103,7 @@ def _create_learning_rate(learning_rate_config):
   if learning_rate_type == 'exponential_decay_learning_rate':
     config = learning_rate_config.exponential_decay_learning_rate
     learning_rate = learning_schedules.exponential_decay_with_burnin(
-        tf.train.get_or_create_global_step(),
+        global_step,
         config.initial_learning_rate,
         config.decay_steps,
         config.decay_factor,
@@ -111,13 +120,13 @@ def _create_learning_rate(learning_rate_config):
     learning_rate_sequence = [config.initial_learning_rate]
     learning_rate_sequence += [x.learning_rate for x in config.schedule]
     learning_rate = learning_schedules.manual_stepping(
-        tf.train.get_or_create_global_step(), learning_rate_step_boundaries,
+        global_step, learning_rate_step_boundaries,
         learning_rate_sequence, config.warmup)
 
   if learning_rate_type == 'cosine_decay_learning_rate':
     config = learning_rate_config.cosine_decay_learning_rate
     learning_rate = learning_schedules.cosine_decay_with_warmup(
-        tf.train.get_or_create_global_step(),
+        global_step,
         config.learning_rate_base,
         config.total_steps,
         config.warmup_learning_rate,

research/object_detection/builders/post_processing_builder.py

@@ -85,14 +85,15 @@ def _build_non_max_suppressor(nms_config):
   if nms_config.max_detections_per_class > nms_config.max_total_detections:
     raise ValueError('max_detections_per_class should be no greater than '
                      'max_total_detections.')
-
   non_max_suppressor_fn = functools.partial(
       post_processing.batch_multiclass_non_max_suppression,
       score_thresh=nms_config.score_threshold,
       iou_thresh=nms_config.iou_threshold,
       max_size_per_class=nms_config.max_detections_per_class,
       max_total_size=nms_config.max_total_detections,
-      use_static_shapes=nms_config.use_static_shapes)
+      use_static_shapes=nms_config.use_static_shapes,
+      use_class_agnostic_nms=nms_config.use_class_agnostic_nms,
+      max_classes_per_detection=nms_config.max_classes_per_detection)
   return non_max_suppressor_fn
 
 

research/object_detection/builders/post_processing_builder_test.py

@@ -41,6 +41,31 @@ class PostProcessingBuilderTest(tf.test.TestCase):
     self.assertAlmostEqual(non_max_suppressor.keywords['score_thresh'], 0.7)
     self.assertAlmostEqual(non_max_suppressor.keywords['iou_thresh'], 0.6)
 
+  def test_build_non_max_suppressor_with_correct_parameters_classagnostic_nms(
+      self):
+    post_processing_text_proto = """
+      batch_non_max_suppression {
+        score_threshold: 0.7
+        iou_threshold: 0.6
+        max_detections_per_class: 10
+        max_total_detections: 300
+        use_class_agnostic_nms: True
+        max_classes_per_detection: 1
+      }
+    """
+    post_processing_config = post_processing_pb2.PostProcessing()
+    text_format.Merge(post_processing_text_proto, post_processing_config)
+    non_max_suppressor, _ = post_processing_builder.build(
+        post_processing_config)
+    self.assertEqual(non_max_suppressor.keywords['max_size_per_class'], 10)
+    self.assertEqual(non_max_suppressor.keywords['max_total_size'], 300)
+    self.assertEqual(non_max_suppressor.keywords['max_classes_per_detection'],
+                     1)
+    self.assertEqual(non_max_suppressor.keywords['use_class_agnostic_nms'],
+                     True)
+    self.assertAlmostEqual(non_max_suppressor.keywords['score_thresh'], 0.7)
+    self.assertAlmostEqual(non_max_suppressor.keywords['iou_thresh'], 0.6)
+
   def test_build_identity_score_converter(self):
     post_processing_text_proto = """
       score_converter: IDENTITY

research/object_detection/builders/preprocessor_builder.py

@@ -39,7 +39,7 @@ def _get_step_config_from_proto(preprocessor_step_config, step_name):
     if field.name == step_name:
       return value
 
-  raise ValueError('Could not get field %s from proto!', step_name)
+  raise ValueError('Could not get field %s from proto!' % step_name)
 
 
 def _get_dict_from_proto(config):
@@ -194,7 +194,7 @@ def build(preprocessor_step_config):
       if len(pad_color) != 3:
         tf.logging.warn('pad_color should have 3 elements (RGB) if set!')
 
-      pad_color = tf.to_float([x for x in config.pad_color])
+      pad_color = tf.cast([x for x in config.pad_color], dtype=tf.float32)
     return (preprocessor.random_pad_image,
             {
                 'min_image_size': min_image_size,
@@ -213,7 +213,7 @@ def build(preprocessor_step_config):
       if len(pad_color) != 3:
         tf.logging.warn('pad_color should have 3 elements (RGB) if set!')
 
-      pad_color = tf.to_float([x for x in config.pad_color])
+      pad_color = tf.cast([x for x in config.pad_color], dtype=tf.float32)
 
     return (preprocessor.random_absolute_pad_image,
             {
@@ -234,7 +234,7 @@ def build(preprocessor_step_config):
       if len(pad_color) != 3:
         tf.logging.warn('pad_color should have 3 elements (RGB) if set!')
 
-      pad_color = tf.to_float([x for x in config.pad_color])
+      pad_color = tf.cast([x for x in config.pad_color], dtype=tf.float32)
 
     kwargs = {
         'min_object_covered': config.min_object_covered,

research/object_detection/core/balanced_positive_negative_sampler.py

@@ -247,7 +247,7 @@ class BalancedPositiveNegativeSampler(minibatch_sampler.MinibatchSampler):
 
         # Sample positive and negative samples separately
         if batch_size is None:
-          max_num_pos = tf.reduce_sum(tf.to_int32(positive_idx))
+          max_num_pos = tf.reduce_sum(tf.cast(positive_idx, dtype=tf.int32))
         else:
           max_num_pos = int(self._positive_fraction * batch_size)
         sampled_pos_idx = self.subsample_indicator(positive_idx, max_num_pos)
@@ -255,8 +255,10 @@ class BalancedPositiveNegativeSampler(minibatch_sampler.MinibatchSampler):
         if batch_size is None:
           negative_positive_ratio = (
               1 - self._positive_fraction) / self._positive_fraction
-          max_num_neg = tf.to_int32(
-              negative_positive_ratio * tf.to_float(num_sampled_pos))
+          max_num_neg = tf.cast(
+              negative_positive_ratio *
+              tf.cast(num_sampled_pos, dtype=tf.float32),
+              dtype=tf.int32)
         else:
           max_num_neg = batch_size - num_sampled_pos
         sampled_neg_idx = self.subsample_indicator(negative_idx, max_num_neg)

research/object_detection/core/box_coder.py

@@ -32,6 +32,8 @@ from abc import abstractproperty
 
 import tensorflow as tf
 
+from object_detection.utils import shape_utils
+
 
 # Box coder types.
 FASTER_RCNN = 'faster_rcnn'
@@ -137,11 +139,12 @@ def batch_decode(encoded_boxes, box_coder, anchors):
     inconsistent.
   """
   encoded_boxes.get_shape().assert_has_rank(3)
-  if encoded_boxes.get_shape()[1].value != anchors.num_boxes_static():
+  if (shape_utils.get_dim_as_int(encoded_boxes.get_shape()[1])
+      != anchors.num_boxes_static()):
     raise ValueError('The number of anchors inferred from encoded_boxes'
                      ' and anchors are inconsistent: shape[1] of encoded_boxes'
                      ' %s should be equal to the number of anchors: %s.' %
-                     (encoded_boxes.get_shape()[1].value,
+                     (shape_utils.get_dim_as_int(encoded_boxes.get_shape()[1]),
                       anchors.num_boxes_static()))
 
   decoded_boxes = tf.stack([

research/object_detection/core/box_list.py

@@ -36,6 +36,8 @@ Some other notes:
 
 import tensorflow as tf
 
+from object_detection.utils import shape_utils
+
 
 class BoxList(object):
   """Box collection."""
@@ -73,7 +75,7 @@ class BoxList(object):
       Number of boxes held in collection (integer) or None if this is not
         inferrable at graph construction time.
     """
-    return self.data['boxes'].get_shape()[0].value
+    return shape_utils.get_dim_as_int(self.data['boxes'].get_shape()[0])
 
   def get_all_fields(self):
     """Returns all fields."""

research/object_detection/core/box_list_ops.py

@@ -339,7 +339,7 @@ def prune_non_overlapping_boxes(
     ioa_ = ioa(boxlist2, boxlist1)  # [M, N] tensor
     ioa_ = tf.reduce_max(ioa_, reduction_indices=[0])  # [N] tensor
     keep_bool = tf.greater_equal(ioa_, tf.constant(min_overlap))
-    keep_inds = tf.squeeze(tf.where(keep_bool), squeeze_dims=[1])
+    keep_inds = tf.squeeze(tf.where(keep_bool), axis=[1])
     new_boxlist1 = gather(boxlist1, keep_inds)
     return new_boxlist1, keep_inds
 
@@ -457,7 +457,7 @@ def boolean_mask(boxlist, indicator, fields=None, scope=None,
     if use_static_shapes:
       if not (indicator_sum and isinstance(indicator_sum, int)):
         raise ValueError('`indicator_sum` must be a of type int')
-      selected_positions = tf.to_float(indicator)
+      selected_positions = tf.cast(indicator, dtype=tf.float32)
       indexed_positions = tf.cast(
           tf.multiply(
               tf.cumsum(selected_positions), selected_positions),
@@ -466,7 +466,7 @@ def boolean_mask(boxlist, indicator, fields=None, scope=None,
           indexed_positions - 1, indicator_sum, dtype=tf.float32)
       sampled_indices = tf.cast(
           tf.tensordot(
-              tf.to_float(tf.range(tf.shape(indicator)[0])),
+              tf.cast(tf.range(tf.shape(indicator)[0]), dtype=tf.float32),
               one_hot_selector,
               axes=[0, 0]),
           dtype=tf.int32)
@@ -962,7 +962,7 @@ def box_voting(selected_boxes, pool_boxes, iou_thresh=0.5):
     raise ValueError('pool_boxes must have a \'scores\' field')
 
   iou_ = iou(selected_boxes, pool_boxes)
-  match_indicator = tf.to_float(tf.greater(iou_, iou_thresh))
+  match_indicator = tf.cast(tf.greater(iou_, iou_thresh), dtype=tf.float32)
   num_matches = tf.reduce_sum(match_indicator, 1)
   # TODO(kbanoop): Handle the case where some boxes in selected_boxes do not
   # match to any boxes in pool_boxes. For such boxes without any matches, we

research/object_detection/core/box_list_ops_test.py

@@ -581,8 +581,8 @@ class BoxListOpsTest(test_case.TestCase):
                            [0, 0, 3, 2]], tf.float32)
     boxes = box_list.BoxList(corners)
     image_and_boxes = box_list_ops.visualize_boxes_in_image(image, boxes)
-    image_and_boxes_bw = tf.to_float(
-        tf.greater(tf.reduce_sum(image_and_boxes, 2), 0.0))
+    image_and_boxes_bw = tf.cast(
+        tf.greater(tf.reduce_sum(image_and_boxes, 2), 0.0), dtype=tf.float32)
     exp_result = [[1, 1, 1, 0],
                   [1, 1, 1, 0],
                   [1, 1, 1, 0],

research/object_detection/core/class_agnostic_nms_test.py

+# Copyright 2019 The TensorFlow Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ==============================================================================
+"""Tests for google3.third_party.tensorflow_models.object_detection.core.class_agnostic_nms."""
+import tensorflow as tf
+from object_detection.core import post_processing
+from object_detection.core import standard_fields as fields
+from object_detection.utils import test_case
+
+
+class ClassAgnosticNonMaxSuppressionTest(test_case.TestCase):
+
+  def test_class_agnostic_nms_select_with_shared_boxes(self):
+    boxes = tf.constant(
+        [[[0, 0, 1, 1]], [[0, 0.1, 1, 1.1]], [[0, -0.1, 1, 0.9]],
+         [[0, 10, 1, 11]], [[0, 10.1, 1, 11.1]], [[0, 100, 1, 101]],
+         [[0, 1000, 1, 1002]], [[0, 1000, 1, 1002.1]]], tf.float32)
+    scores = tf.constant([[.9, 0.01], [.75, 0.05], [.6, 0.01], [.95, 0],
+                          [.5, 0.01], [.3, 0.01], [.01, .85], [.01, .5]])
+    score_thresh = 0.1
+    iou_thresh = .5
+    max_classes_per_detection = 1
+    max_output_size = 4
+
+    exp_nms_corners = [[0, 10, 1, 11], [0, 0, 1, 1], [0, 1000, 1, 1002],
+                       [0, 100, 1, 101]]
+    exp_nms_scores = [.95, .9, .85, .3]
+    exp_nms_classes = [0, 0, 1, 0]
+
+    nms, _ = post_processing.class_agnostic_non_max_suppression(
+        boxes, scores, score_thresh, iou_thresh, max_classes_per_detection,
+        max_output_size)
+
+    with self.test_session() as sess:
+      nms_corners_output, nms_scores_output, nms_classes_output = sess.run([
+          nms.get(),
+          nms.get_field(fields.BoxListFields.scores),
+          nms.get_field(fields.BoxListFields.classes)
+      ])
+
+      self.assertAllClose(nms_corners_output, exp_nms_corners)
+      self.assertAllClose(nms_scores_output, exp_nms_scores)
+      self.assertAllClose(nms_classes_output, exp_nms_classes)
+
+  def test_class_agnostic_nms_select_with_per_class_boxes(self):
+    boxes = tf.constant(
+        [[[4, 5, 9, 10], [0, 0, 1, 1]],
+         [[0, 0.1, 1, 1.1], [4, 5, 9, 10]],
+         [[0, -0.1, 1, 0.9], [4, 5, 9, 10]],
+         [[0, 10, 1, 11], [4, 5, 9, 10]],
+         [[0, 10.1, 1, 11.1], [4, 5, 9, 10]],
+         [[0, 100, 1, 101], [4, 5, 9, 10]],
+         [[4, 5, 9, 10], [0, 1000, 1, 1002]],
+         [[4, 5, 9, 10], [0, 1000, 1, 1002.1]]], tf.float32)
+    scores = tf.constant([[.01, 0.9],
+                          [.75, 0.05],
+                          [.6, 0.01],
+                          [.95, 0],
+                          [.5, 0.01],
+                          [.3, 0.01],
+                          [.01, .85],
+                          [.01, .5]])
+    score_thresh = 0.1
+    iou_thresh = .5
+    max_classes_per_detection = 1
+    max_output_size = 4
+
+    exp_nms_corners = [[0, 10, 1, 11],
+                       [0, 0, 1, 1],
+                       [0, 1000, 1, 1002],
+                       [0, 100, 1, 101]]
+    exp_nms_scores = [.95, .9, .85, .3]
+    exp_nms_classes = [0, 1, 1, 0]
+
+    nms, _ = post_processing.class_agnostic_non_max_suppression(
+        boxes, scores, score_thresh, iou_thresh, max_classes_per_detection,
+        max_output_size)
+
+    with self.test_session() as sess:
+      nms_corners_output, nms_scores_output, nms_classes_output = sess.run([
+          nms.get(),
+          nms.get_field(fields.BoxListFields.scores),
+          nms.get_field(fields.BoxListFields.classes)
+      ])
+
+      self.assertAllClose(nms_corners_output, exp_nms_corners)
+      self.assertAllClose(nms_scores_output, exp_nms_scores)
+      self.assertAllClose(nms_classes_output, exp_nms_classes)
+
+  def test_batch_classagnostic_nms_with_batch_size_1(self):
+    boxes = tf.constant(
+        [[[[0, 0, 1, 1]], [[0, 0.1, 1, 1.1]], [[0, -0.1, 1, 0.9]],
+          [[0, 10, 1, 11]], [[0, 10.1, 1, 11.1]], [[0, 100, 1, 101]],
+          [[0, 1000, 1, 1002]], [[0, 1000, 1, 1002.1]]]], tf.float32)
+    scores = tf.constant([[[.9, 0.01], [.75, 0.05], [.6, 0.01], [.95, 0],
+                           [.5, 0.01], [.3, 0.01], [.01, .85], [.01, .5]]])
+    score_thresh = 0.1
+    iou_thresh = .5
+    max_output_size = 4
+    max_classes_per_detection = 1
+    use_class_agnostic_nms = True
+
+    exp_nms_corners = [[[0, 10, 1, 11], [0, 0, 1, 1], [0, 1000, 1, 1002],
+                        [0, 100, 1, 101]]]
+    exp_nms_scores = [[.95, .9, .85, .3]]
+    exp_nms_classes = [[0, 0, 1, 0]]
+
+    (nmsed_boxes, nmsed_scores, nmsed_classes, nmsed_masks,
+     nmsed_additional_fields,
+     num_detections) = post_processing.batch_multiclass_non_max_suppression(
+         boxes,
+         scores,
+         score_thresh,
+         iou_thresh,
+         max_size_per_class=max_output_size,
+         max_total_size=max_output_size,
+         use_class_agnostic_nms=use_class_agnostic_nms,
+         max_classes_per_detection=max_classes_per_detection)
+
+    self.assertIsNone(nmsed_masks)
+    self.assertIsNone(nmsed_additional_fields)
+
+    with self.test_session() as sess:
+      (nmsed_boxes, nmsed_scores, nmsed_classes, num_detections) = sess.run(
+          [nmsed_boxes, nmsed_scores, nmsed_classes, num_detections])
+      self.assertAllClose(nmsed_boxes, exp_nms_corners)
+      self.assertAllClose(nmsed_scores, exp_nms_scores)
+      self.assertAllClose(nmsed_classes, exp_nms_classes)
+      self.assertEqual(num_detections, [4])
+
+
+if __name__ == '__main__':
+  tf.test.main()

research/object_detection/core/freezable_batch_norm.py

@@ -36,12 +36,9 @@ class FreezableBatchNorm(tf.keras.layers.BatchNormalization):
   close to 0 and the activation standard deviation close to 1.
 
   Arguments:
-    training: Boolean or None. If True, the batch normalization layer will
-      normalize the input batch using the batch mean and standard deviation,
-      and update the total moving mean and standard deviations. If False, the
-      layer will normalize using the moving average and std. dev, without
-      updating the learned avg and std. dev.
-      If None, the layer will follow the keras BatchNormalization layer
+    training: If False, the layer will normalize using the moving average and
+      std. dev, without updating the learned avg and std. dev.
+      If None or True, the layer will follow the keras BatchNormalization layer
       strategy of checking the Keras learning phase at `call` time to decide
       what to do.
     **kwargs: The keyword arguments to forward to the keras BatchNormalization
@@ -65,6 +62,7 @@ class FreezableBatchNorm(tf.keras.layers.BatchNormalization):
     self._training = training
 
   def call(self, inputs, training=None):
-    if training is None:
+    # Override the call arg only if the batchnorm is frozen. (Ignore None)
+    if self._training is False:  # pylint: disable=g-bool-id-comparison
       training = self._training
     return super(FreezableBatchNorm, self).call(inputs, training=training)

research/object_detection/core/freezable_batch_norm_test.py

@@ -43,7 +43,24 @@ class FreezableBatchNormTest(tf.test.TestCase):
     model.fit(train_data, train_data, epochs=4, verbose=0)
     return model.weights
 
-  def test_batchnorm_freezing_training_true(self):
+  def _test_batchnorm_layer(
+      self, norm, should_be_training, test_data,
+      testing_mean, testing_var, training_arg, training_mean, training_var):
+    out_tensor = norm(tf.convert_to_tensor(test_data, dtype=tf.float32),
+                      training=training_arg)
+    out = tf.keras.backend.eval(out_tensor)
+    out -= tf.keras.backend.eval(norm.beta)
+    out /= tf.keras.backend.eval(norm.gamma)
+
+    if not should_be_training:
+      out *= training_var
+      out += (training_mean - testing_mean)
+      out /= testing_var
+
+    np.testing.assert_allclose(out.mean(), 0.0, atol=1.5e-1)
+    np.testing.assert_allclose(out.std(), 1.0, atol=1.5e-1)
+
+  def test_batchnorm_freezing_training_none(self):
     with self.test_session():
       training_mean = 5.0
       training_var = 10.0
@@ -69,14 +86,38 @@ class FreezableBatchNormTest(tf.test.TestCase):
           scale=testing_var,
           size=(1000, 10))
 
-      out_tensor = norm(tf.convert_to_tensor(test_data, dtype=tf.float32))
-      out = tf.keras.backend.eval(out_tensor)
-
-      out -= tf.keras.backend.eval(norm.beta)
-      out /= tf.keras.backend.eval(norm.gamma)
-
-      np.testing.assert_allclose(out.mean(), 0.0, atol=1.5e-1)
-      np.testing.assert_allclose(out.std(), 1.0, atol=1.5e-1)
+      # Test with training=True passed to the call method:
+      training_arg = True
+      should_be_training = True
+      self._test_batchnorm_layer(norm, should_be_training, test_data,
+                                 testing_mean, testing_var, training_arg,
+                                 training_mean, training_var)
+
+      # Test with training=False passed to the call method:
+      training_arg = False
+      should_be_training = False
+      self._test_batchnorm_layer(norm, should_be_training, test_data,
+                                 testing_mean, testing_var, training_arg,
+                                 training_mean, training_var)
+
+      # Test the layer in various Keras learning phase scopes:
+      training_arg = None
+      should_be_training = False
+      self._test_batchnorm_layer(norm, should_be_training, test_data,
+                                 testing_mean, testing_var, training_arg,
+                                 training_mean, training_var)
+
+      tf.keras.backend.set_learning_phase(True)
+      should_be_training = True
+      self._test_batchnorm_layer(norm, should_be_training, test_data,
+                                 testing_mean, testing_var, training_arg,
+                                 training_mean, training_var)
+
+      tf.keras.backend.set_learning_phase(False)
+      should_be_training = False
+      self._test_batchnorm_layer(norm, should_be_training, test_data,
+                                 testing_mean, testing_var, training_arg,
+                                 training_mean, training_var)
 
   def test_batchnorm_freezing_training_false(self):
     with self.test_session():
@@ -104,18 +145,40 @@ class FreezableBatchNormTest(tf.test.TestCase):
           scale=testing_var,
           size=(1000, 10))
 
-      out_tensor = norm(tf.convert_to_tensor(test_data, dtype=tf.float32))
-      out = tf.keras.backend.eval(out_tensor)
-
-      out -= tf.keras.backend.eval(norm.beta)
-      out /= tf.keras.backend.eval(norm.gamma)
-
-      out *= training_var
-      out += (training_mean - testing_mean)
-      out /= testing_var
+      # Make sure that the layer is never training
+      # Test with training=True passed to the call method:
+      training_arg = True
+      should_be_training = False
+      self._test_batchnorm_layer(norm, should_be_training, test_data,
+                                 testing_mean, testing_var, training_arg,
+                                 training_mean, training_var)
+
+      # Test with training=False passed to the call method:
+      training_arg = False
+      should_be_training = False
+      self._test_batchnorm_layer(norm, should_be_training, test_data,
+                                 testing_mean, testing_var, training_arg,
+                                 training_mean, training_var)
+
+      # Test the layer in various Keras learning phase scopes:
+      training_arg = None
+      should_be_training = False
+      self._test_batchnorm_layer(norm, should_be_training, test_data,
+                                 testing_mean, testing_var, training_arg,
+                                 training_mean, training_var)
+
+      tf.keras.backend.set_learning_phase(True)
+      should_be_training = False
+      self._test_batchnorm_layer(norm, should_be_training, test_data,
+                                 testing_mean, testing_var, training_arg,
+                                 training_mean, training_var)
+
+      tf.keras.backend.set_learning_phase(False)
+      should_be_training = False
+      self._test_batchnorm_layer(norm, should_be_training, test_data,
+                                 testing_mean, testing_var, training_arg,
+                                 training_mean, training_var)
 
-      np.testing.assert_allclose(out.mean(), 0.0, atol=1.5e-1)
-      np.testing.assert_allclose(out.std(), 1.0, atol=1.5e-1)
 
 if __name__ == '__main__':
   tf.test.main()

research/object_detection/core/losses.py

@@ -26,9 +26,7 @@ Classification losses:
  * WeightedSoftmaxClassificationAgainstLogitsLoss
  * BootstrappedSigmoidClassificationLoss
 """
-from abc import ABCMeta
-from abc import abstractmethod
-
+import abc
 import tensorflow as tf
 
 from object_detection.core import box_list
@@ -40,7 +38,7 @@ slim = tf.contrib.slim
 
 class Loss(object):
   """Abstract base class for loss functions."""
-  __metaclass__ = ABCMeta
+  __metaclass__ = abc.ABCMeta
 
   def __call__(self,
                prediction_tensor,
@@ -96,7 +94,7 @@ class Loss(object):
     loss_multiplier_shape = tf.stack([-1] + [1] * (len(tensor.shape) - 1))
     return tf.cast(tf.reshape(losses_mask, loss_multiplier_shape), tf.float32)
 
-  @abstractmethod
+  @abc.abstractmethod
   def _compute_loss(self, prediction_tensor, target_tensor, **params):
     """Method to be overridden by implementations.
 
@@ -616,8 +614,10 @@ class HardExampleMiner(object):
   def summarize(self):
     """Summarize the number of positives and negatives after mining."""
     if self._num_positives_list and self._num_negatives_list:
-      avg_num_positives = tf.reduce_mean(tf.to_float(self._num_positives_list))
-      avg_num_negatives = tf.reduce_mean(tf.to_float(self._num_negatives_list))
+      avg_num_positives = tf.reduce_mean(
+          tf.cast(self._num_positives_list, dtype=tf.float32))
+      avg_num_negatives = tf.reduce_mean(
+          tf.cast(self._num_negatives_list, dtype=tf.float32))
       tf.summary.scalar('HardExampleMiner/NumPositives', avg_num_positives)
       tf.summary.scalar('HardExampleMiner/NumNegatives', avg_num_negatives)
 
@@ -661,12 +661,13 @@ class HardExampleMiner(object):
     """
     positives_indicator = tf.gather(match.matched_column_indicator(), indices)
     negatives_indicator = tf.gather(match.unmatched_column_indicator(), indices)
-    num_positives = tf.reduce_sum(tf.to_int32(positives_indicator))
-    max_negatives = tf.maximum(min_negatives_per_image,
-                               tf.to_int32(max_negatives_per_positive *
-                                           tf.to_float(num_positives)))
+    num_positives = tf.reduce_sum(tf.cast(positives_indicator, dtype=tf.int32))
+    max_negatives = tf.maximum(
+        min_negatives_per_image,
+        tf.cast(max_negatives_per_positive *
+                tf.cast(num_positives, dtype=tf.float32), dtype=tf.int32))
     topk_negatives_indicator = tf.less_equal(
-        tf.cumsum(tf.to_int32(negatives_indicator)), max_negatives)
+        tf.cumsum(tf.cast(negatives_indicator, dtype=tf.int32)), max_negatives)
     subsampled_selection_indices = tf.where(
         tf.logical_or(positives_indicator, topk_negatives_indicator))
     num_negatives = tf.size(subsampled_selection_indices) - num_positives

research/object_detection/core/matcher.py

@@ -31,9 +31,7 @@ consider this box a positive example (match) nor a negative example (no match).
 The Match class is used to store the match results and it provides simple apis
 to query the results.
 """
-from abc import ABCMeta
-from abc import abstractmethod
-
+import abc
 import tensorflow as tf
 
 from object_detection.utils import ops
@@ -170,7 +168,7 @@ class Match(object):
       row_indices: int32 tensor of shape [K] with row indices.
     """
     return self._reshape_and_cast(
-        self._gather_op(tf.to_float(self._match_results),
+        self._gather_op(tf.cast(self._match_results, dtype=tf.float32),
                         self.matched_column_indices()))
 
   def num_matched_rows(self):
@@ -215,7 +213,7 @@ class Match(object):
 class Matcher(object):
   """Abstract base class for matcher.
   """
-  __metaclass__ = ABCMeta
+  __metaclass__ = abc.ABCMeta
 
   def __init__(self, use_matmul_gather=False):
     """Constructs a Matcher.
@@ -249,7 +247,7 @@ class Matcher(object):
       return Match(self._match(similarity_matrix, valid_rows),
                    self._use_matmul_gather)
 
-  @abstractmethod
+  @abc.abstractmethod
   def _match(self, similarity_matrix, valid_rows):
     """Method to be overridden by implementations.