Skip to content

GitLab

Commit 05584085 authored by pkulzc's avatar pkulzc Committed by Jonathan Huang
Browse files

Merged commit includes the following changes: (#6315) 

236813471  by lzc:

    Internal change.

--
236507310  by lzc:

    Fix preprocess.random_resize_method config type issue. The target height and width will be passed as "size" to tf.image.resize_images which only accepts integer.

--
236409989  by Zhichao Lu:

    Config export_to_tpu from function parameter instead of HParams for TPU inference.

--
236403186  by Zhichao Lu:

    Make graph file names optional arguments.

--
236237072  by Zhichao Lu:

    Minor bugfix for keyword args.

--
236209602  by Zhichao Lu:

    Add support for PartitionedVariable to get_variables_available_in_checkpoint.

--
235828658  by Zhichao Lu:

    Automatically stop evaluation jobs when training is finished.

--
235817964  by Zhichao Lu:

    Add an optional process_metrics_fn callback to eval_util, it gets called
    with evaluation results once each evaluation is complete.

--
235788721  by lzc:

    Fix yml file tf runtime version.

--
235262897  by Zhichao Lu:

    Add keypoint support to the random_pad_image preprocessor method.

--
235257380  by Zhichao Lu:

    Support InputDataFields.groundtruth_confidences in retain_groundtruth(), retain_groundtruth_with_positive_classes(), filter_groundtruth_with_crowd_boxes(), filter_groundtruth_with_nan_box_coordinates(), filter_unrecognized_classes().

--
235109188  by Zhichao Lu:

    Fix bug in pad_input_data_to_static_shapes for num_additional_channels > 0; make color-specific data augmentation only touch RGB channels.

--
235045010  by Zhichao Lu:

    Don't slice class_predictions_with_background when add_background_class is false.

--
235026189  by lzc:

    Fix import in g3doc.

--
234863426  by Zhichao Lu:

    Added fixes in exporter to allow writing a checkpoint to a specified temporary directory.

--
234671886  by lzc:

    Internal Change.

--
234630803  by rathodv:

    Internal Change.

--
233985896  by Zhichao Lu:

    Add Neumann optimizer to object detection.

--
233560911  by Zhichao Lu:

    Add NAS-FPN object detection with Resnet and Mobilenet v2.

--
233513536  by Zhichao Lu:

    Export TPU compatible object detection model

--
233495772  by lzc:

    Internal change.

--
233453557  by Zhichao Lu:

    Create Keras-based SSD+MobilenetV1 for object detection.

--
233220074  by lzc:

    Update release notes date.

--
233165761  by Zhichao Lu:

    Support depth_multiplier and min_depth in _SSDResnetV1FpnFeatureExtractor.

--
233160046  by lzc:

    Internal change.

--
232926599  by Zhichao Lu:

    [tf.data] Switching tf.data functions to use `defun`, providing an escape hatch to continue using the legacy `Defun`.

    There are subtle differences between the implementation of `defun` and `Defun` (such as resources handling or control flow) and it is possible that input pipelines that use control flow or resources in their functions might be affected by this change. To migrate majority of existing pipelines to the recommended way of creating functions in TF 2.0 world, while allowing (a small number of) existing pipelines to continue relying on the deprecated behavior, this CL provides an escape hatch.

    If your input pipeline is affected by this CL, it should apply the escape hatch by replacing `foo.map(...)` with `foo.map_with_legacy_function(...)`.

--
232891621  by Zhichao Lu:

    Modify faster_rcnn meta architecture to normalize raw detections.

--
232875817  by Zhichao Lu:

    Make calibration a post-processing step.

    Specifically:
    - Move the calibration config from pipeline.proto --> post_processing.proto
    - Edit post_processing_builder.py to return a calibration function. If no calibration config is provided, it None.
    - Edit SSD and FasterRCNN meta architectures to optionally call the calibration function on detection scores after score conversion and before NMS.

--
232704481  by Zhichao Lu:

    Edit calibration builder to build a function that will be used within a detection model's `postprocess` method, after score conversion and before non-maxima suppression.

    Specific Edits:
    - The returned function now accepts class_predictions_with_background as its argument instead of detection_scores and detection_classes.
    - Class-specific calibration was temporarily removed, as it requires more significant refactoring. Will be added later.

--
232615379  by Zhichao Lu:

    Internal change

--
232483345  by ronnyvotel:

    Making the use of bfloat16 restricted to TPUs.

--
232399572  by Zhichao Lu:

    Edit calibration builder and proto to support class-agnostic calibration.

    Specifically:
    - Edit calibration protos to include path to relevant label map if required for class-specific calibration. Previously, label maps were inferred from other parts of the pipeline proto; this allows all information required by the builder stay within the calibration proto and remove extraneous information from being passed with class-agnostic calibration.
    - Add class-agnostic protos to the calibration config.

    Note that the proto supports sigmoid and linear interpolation parameters, but the builder currently only supports linear interpolation.

--
231613048  by Zhichao Lu:

    Add calibration builder for applying calibration transformations from output of object detection models.

    Specifically:
    - Add calibration proto to support sigmoid and isotonic regression (stepwise function) calibration.
    - Add a builder to support calibration from isotonic regression outputs.

--
231519786  by lzc:

    model_builder test refactor.
    - removed proto text boilerplate in each test case and let them call a create_default_proto function instead.
    - consolidated all separate ssd model creation tests into one.
    - consolidated all separate faster rcnn model creation tests into one.
    - used parameterized test for testing mask rcnn models and use_matmul_crop_and_resize
    - added all failures test.

--
231448169  by Zhichao Lu:

    Return static shape as a constant tensor.

--
231423126  by lzc:

    Add a release note for OID v4 models.

--
231401941  by Zhichao Lu:

    Adding correct labelmap for the models trained on Open Images V4 (*oid_v4
    config suffix).

--
231320357  by Zhichao Lu:

    Add scope to Nearest Neighbor Resize op so that it stays in the same name scope as the original resize ops.

--
231257699  by Zhichao Lu:

    Switch to using preserve_aspect_ratio in tf.image.resize_images rather than using a custom implementation.

--
231247368  by rathodv:

    Internal change.

--
231004874  by lzc:

    Update documentations to use tf 1.12 for object detection API.

--
230999911  by rathodv:

    Use tf.batch_gather instead of ops.batch_gather

--
230999720  by huizhongc:

    Fix weight equalization test in ops_test.

--
230984728  by rathodv:

    Internal update.

--
230929019  by lzc:

    Add an option to replace preprocess operation with placeholder for ssd feature extractor.

--
230845266  by lzc:

    Require tensorflow version 1.12 for object detection API and rename keras_applications to keras_models

--
230392064  by lzc:

    Add RetinaNet 101 checkpoint trained on OID v4 to detection model zoo.

--
230014128  by derekjchow:

    This file was re-located below the tensorflow/lite/g3doc/convert

--
229941449  by lzc:

    Update SSD mobilenet v2 quantized model download path.

--
229843662  by lzc:

    Add an option to use native resize tf op in fpn top-down feature map generation.

--
229636034  by rathodv:

    Add deprecation notice to a few old parameters in train.proto

--
228959078  by derekjchow:

    Remove duplicate elif case in _check_and_convert_legacy_input_config_key

--
228749719  by rathodv:

    Minor refactoring to make exporter's `build_detection_graph` method public.

--
228573828  by rathodv:

    Mofity model.postprocess to return raw detections and raw scores.

    Modify, post-process methods in core/model.py and the meta architectures to export raw detection (without any non-max suppression) and raw multiclass score logits for those detections.

--
228420670  by Zhichao Lu:

    Add shims for custom architectures for object detection models.

--
228241692  by Zhichao Lu:

    Fix the comment on "losses_mask" in "Loss" class.

--
228223810  by Zhichao Lu:

    Support other_heads' predictions in WeightSharedConvolutionalBoxPredictor. Also remove a few unused parameters and fix a couple of comments in convolutional_box_predictor.py.

--
228200588  by Zhichao Lu:

    Add Expected Calibration Error and an evaluator that calculates the metric for object detections.

--
228167740  by lzc:

    Add option to use bounded activations in FPN top-down feature map generation.

--
227767700  by rathodv:

    Internal.

--
226295236  by Zhichao Lu:

    Add Open Image V4 Resnet101-FPN training config to third_party

--
226254842  by Zhichao Lu:

    Fix typo in documentation.

--
225833971  by Zhichao Lu:

    Option to have no resizer in object detection model.

--
225824890  by lzc:

    Fixes p3 compatibility for model_lib.py

--
225760897  by menglong:

    normalizer should be at least 1.

--
225559842  by menglong:

    Add extra logic filtering unrecognized classes.

--
225379421  by lzc:

    Add faster_rcnn_inception_resnet_v2_atrous_oid_v4 config to third_party

--
225368337  by Zhichao Lu:

    Add extra logic filtering unrecognized classes.

--
225341095  by Zhichao Lu:

    Adding Open Images V4 models to OD API model zoo and corresponding configs to the
    configs.

--
225218450  by menglong:

    Add extra logic filtering unrecognized classes.

--
225057591  by Zhichao Lu:

    Internal change.

--
224895417  by rathodv:

    Internal change.

--
224209282  by Zhichao Lu:

    Add two data augmentations to object detection: (1) Self-concat (2) Absolute pads.

--
224073762  by Zhichao Lu:

    Do not create tf.constant until _generate() is actually called in the object detector.

--

PiperOrigin-RevId: 236813471
parent a5db4420
Hide whitespace changes
Inline Side-by-side
Showing with 3889 additions and 166 deletions
research/object_detection/core/preprocessor.py
View file @ 05584085
......@@ -826,6 +826,14 @@ def random_image_scale(image,
return tuple(result)
def _augment_only_rgb_channels(image, augment_function):
"""Augments only the RGB slice of an image with additional channels."""
rgb_slice = image[:, :, :3]
augmented_rgb_slice = augment_function(rgb_slice)
image = tf.concat([augmented_rgb_slice, image[:, :, 3:]], -1)
return image
def random_rgb_to_gray(image,
probability=0.1,
seed=None,
......@@ -860,7 +868,7 @@ def random_rgb_to_gray(image,
image = tf.cond(
tf.greater(do_gray_random, probability), lambda: image,
lambda: _image_to_gray(image))
lambda: _augment_only_rgb_channels(image, _image_to_gray))
return image
......@@ -895,8 +903,12 @@ def random_adjust_brightness(image,
preprocessor_cache.PreprocessorCache.ADJUST_BRIGHTNESS,
preprocess_vars_cache)
image = tf.image.adjust_brightness(image / 255, delta) * 255
image = tf.clip_by_value(image, clip_value_min=0.0, clip_value_max=255.0)
def _adjust_brightness(image):
image = tf.image.adjust_brightness(image / 255, delta) * 255
image = tf.clip_by_value(image, clip_value_min=0.0, clip_value_max=255.0)
return image
image = _augment_only_rgb_channels(image, _adjust_brightness)
return image
......@@ -932,8 +944,12 @@ def random_adjust_contrast(image,
generator_func,
preprocessor_cache.PreprocessorCache.ADJUST_CONTRAST,
preprocess_vars_cache)
image = tf.image.adjust_contrast(image / 255, contrast_factor) * 255
image = tf.clip_by_value(image, clip_value_min=0.0, clip_value_max=255.0)
def _adjust_contrast(image):
image = tf.image.adjust_contrast(image / 255, contrast_factor) * 255
image = tf.clip_by_value(image, clip_value_min=0.0, clip_value_max=255.0)
return image
image = _augment_only_rgb_channels(image, _adjust_contrast)
return image
......@@ -964,8 +980,11 @@ def random_adjust_hue(image,
delta = _get_or_create_preprocess_rand_vars(
generator_func, preprocessor_cache.PreprocessorCache.ADJUST_HUE,
preprocess_vars_cache)
image = tf.image.adjust_hue(image / 255, delta) * 255
image = tf.clip_by_value(image, clip_value_min=0.0, clip_value_max=255.0)
def _adjust_hue(image):
image = tf.image.adjust_hue(image / 255, delta) * 255
image = tf.clip_by_value(image, clip_value_min=0.0, clip_value_max=255.0)
return image
image = _augment_only_rgb_channels(image, _adjust_hue)
return image
......@@ -1001,8 +1020,11 @@ def random_adjust_saturation(image,
generator_func,
preprocessor_cache.PreprocessorCache.ADJUST_SATURATION,
preprocess_vars_cache)
image = tf.image.adjust_saturation(image / 255, saturation_factor) * 255
image = tf.clip_by_value(image, clip_value_min=0.0, clip_value_max=255.0)
def _adjust_saturation(image):
image = tf.image.adjust_saturation(image / 255, saturation_factor) * 255
image = tf.clip_by_value(image, clip_value_min=0.0, clip_value_max=255.0)
return image
image = _augment_only_rgb_channels(image, _adjust_saturation)
return image
......@@ -1423,6 +1445,7 @@ def random_crop_image(image,
def random_pad_image(image,
boxes,
keypoints=None,
min_image_size=None,
max_image_size=None,
pad_color=None,
......@@ -1444,15 +1467,18 @@ def random_pad_image(image,
Boxes are in normalized form meaning their coordinates vary
between [0, 1].
Each row is in the form of [ymin, xmin, ymax, xmax].
keypoints: (optional) rank 3 float32 tensor with shape
[N, num_keypoints, 2]. The keypoints are in y-x normalized
coordinates.
min_image_size: a tensor of size [min_height, min_width], type tf.int32.
If passed as None, will be set to image size
[height, width].
max_image_size: a tensor of size [max_height, max_width], type tf.int32.
If passed as None, will be set to twice the
image [height * 2, width * 2].
pad_color: padding color. A rank 1 tensor of [3] with dtype=tf.float32.
if set as None, it will be set to average color of the input
image.
pad_color: padding color. A rank 1 tensor of [channels] with dtype=
tf.float32. if set as None, it will be set to average color of
the input image.
seed: random seed.
preprocess_vars_cache: PreprocessorCache object that records previously
performed augmentations. Updated in-place. If this
......@@ -1463,6 +1489,9 @@ def random_pad_image(image,
image: Image shape will be [new_height, new_width, channels].
boxes: boxes which is the same rank as input boxes. Boxes are in normalized
form.
if keypoints is not None, the function also returns:
keypoints: rank 3 float32 tensor with shape [N, num_keypoints, 2]
"""
if pad_color is None:
pad_color = tf.reduce_mean(image, axis=[0, 1])
......@@ -1537,7 +1566,62 @@ def random_pad_image(image,
new_boxlist = box_list_ops.change_coordinate_frame(boxlist, new_window)
new_boxes = new_boxlist.get()
return new_image, new_boxes
result = [new_image, new_boxes]
if keypoints is not None:
new_keypoints = keypoint_ops.change_coordinate_frame(keypoints, new_window)
result.append(new_keypoints)
return tuple(result)
def random_absolute_pad_image(image,
boxes,
max_height_padding,
max_width_padding,
pad_color=None,
seed=None,
preprocess_vars_cache=None):
"""Randomly pads the image by small absolute amounts.
As random_pad_image above, but the padding is of size [0, max_height_padding]
or [0, max_width_padding] instead of padding to a fixed size of
max_height_padding for all images.
Args:
image: rank 3 float32 tensor containing 1 image -> [height, width, channels]
with pixel values varying between [0, 1].
boxes: rank 2 float32 tensor containing the bounding boxes -> [N, 4].
Boxes are in normalized form meaning their coordinates vary
between [0, 1].
Each row is in the form of [ymin, xmin, ymax, xmax].
max_height_padding: a scalar tf.int32 tensor denoting the maximum amount of
height padding. The padding will be chosen uniformly at
random from [0, max_height_padding).
max_width_padding: a scalar tf.int32 tensor denoting the maximum amount of
width padding. The padding will be chosen uniformly at
random from [0, max_width_padding).
pad_color: padding color. A rank 1 tensor of [3] with dtype=tf.float32.
if set as None, it will be set to average color of the input
image.
seed: random seed.
preprocess_vars_cache: PreprocessorCache object that records previously
performed augmentations. Updated in-place. If this
function is called multiple times with the same
non-null cache, it will perform deterministically.
Returns:
image: Image shape will be [new_height, new_width, channels].
boxes: boxes which is the same rank as input boxes. Boxes are in normalized
form.
"""
min_image_size = tf.shape(image)[:2]
max_image_size = min_image_size + tf.to_int32(
[max_height_padding, max_width_padding])
return random_pad_image(image, boxes, min_image_size=min_image_size,
max_image_size=max_image_size, pad_color=pad_color,
seed=seed,
preprocess_vars_cache=preprocess_vars_cache)
def random_crop_pad_image(image,
......@@ -2101,80 +2185,6 @@ def random_resize_method(image, target_size, preprocess_vars_cache=None):
return resized_image
def _compute_new_static_size(image, min_dimension, max_dimension):
"""Compute new static shape for resize_to_range method."""
image_shape = image.get_shape().as_list()
orig_height = image_shape[0]
orig_width = image_shape[1]
num_channels = image_shape[2]
orig_min_dim = min(orig_height, orig_width)
# Calculates the larger of the possible sizes
large_scale_factor = min_dimension / float(orig_min_dim)
# Scaling orig_(height|width) by large_scale_factor will make the smaller
# dimension equal to min_dimension, save for floating point rounding errors.
# For reasonably-sized images, taking the nearest integer will reliably
# eliminate this error.
large_height = int(round(orig_height * large_scale_factor))
large_width = int(round(orig_width * large_scale_factor))
large_size = [large_height, large_width]
if max_dimension:
# Calculates the smaller of the possible sizes, use that if the larger
# is too big.
orig_max_dim = max(orig_height, orig_width)
small_scale_factor = max_dimension / float(orig_max_dim)
# Scaling orig_(height|width) by small_scale_factor will make the larger
# dimension equal to max_dimension, save for floating point rounding
# errors. For reasonably-sized images, taking the nearest integer will
# reliably eliminate this error.
small_height = int(round(orig_height * small_scale_factor))
small_width = int(round(orig_width * small_scale_factor))
small_size = [small_height, small_width]
new_size = large_size
if max(large_size) > max_dimension:
new_size = small_size
else:
new_size = large_size
return tf.constant(new_size + [num_channels])
def _compute_new_dynamic_size(image, min_dimension, max_dimension):
"""Compute new dynamic shape for resize_to_range method."""
image_shape = tf.shape(image)
orig_height = tf.to_float(image_shape[0])
orig_width = tf.to_float(image_shape[1])
num_channels = image_shape[2]
orig_min_dim = tf.minimum(orig_height, orig_width)
# Calculates the larger of the possible sizes
min_dimension = tf.constant(min_dimension, dtype=tf.float32)
large_scale_factor = min_dimension / orig_min_dim
# Scaling orig_(height|width) by large_scale_factor will make the smaller
# dimension equal to min_dimension, save for floating point rounding errors.
# For reasonably-sized images, taking the nearest integer will reliably
# eliminate this error.
large_height = tf.to_int32(tf.round(orig_height * large_scale_factor))
large_width = tf.to_int32(tf.round(orig_width * large_scale_factor))
large_size = tf.stack([large_height, large_width])
if max_dimension:
# Calculates the smaller of the possible sizes, use that if the larger
# is too big.
orig_max_dim = tf.maximum(orig_height, orig_width)
max_dimension = tf.constant(max_dimension, dtype=tf.float32)
small_scale_factor = max_dimension / orig_max_dim
# Scaling orig_(height|width) by small_scale_factor will make the larger
# dimension equal to max_dimension, save for floating point rounding
# errors. For reasonably-sized images, taking the nearest integer will
# reliably eliminate this error.
small_height = tf.to_int32(tf.round(orig_height * small_scale_factor))
small_width = tf.to_int32(tf.round(orig_width * small_scale_factor))
small_size = tf.stack([small_height, small_width])
new_size = tf.cond(
tf.to_float(tf.reduce_max(large_size)) > max_dimension,
lambda: small_size, lambda: large_size)
else:
new_size = large_size
return tf.stack(tf.unstack(new_size) + [num_channels])
def resize_to_range(image,
masks=None,
min_dimension=None,
......@@ -2228,13 +2238,31 @@ def resize_to_range(image,
if len(image.get_shape()) != 3:
raise ValueError('Image should be 3D tensor')
def _resize_landscape_image(image):
# resize a landscape image
return tf.image.resize_images(
image, tf.stack([min_dimension, max_dimension]), method=method,
align_corners=align_corners, preserve_aspect_ratio=True)
def _resize_portrait_image(image):
# resize a portrait image
return tf.image.resize_images(
image, tf.stack([max_dimension, min_dimension]), method=method,
align_corners=align_corners, preserve_aspect_ratio=True)
with tf.name_scope('ResizeToRange', values=[image, min_dimension]):
if image.get_shape().is_fully_defined():
new_size = _compute_new_static_size(image, min_dimension, max_dimension)
if image.get_shape()[0] < image.get_shape()[1]:
new_image = _resize_landscape_image(image)
else:
new_image = _resize_portrait_image(image)
new_size = tf.constant(new_image.get_shape().as_list())
else:
new_size = _compute_new_dynamic_size(image, min_dimension, max_dimension)
new_image = tf.image.resize_images(
image, new_size[:-1], method=method, align_corners=align_corners)
new_image = tf.cond(
tf.less(tf.shape(image)[0], tf.shape(image)[1]),
lambda: _resize_landscape_image(image),
lambda: _resize_portrait_image(image))
new_size = tf.shape(new_image)
if pad_to_max_dimension:
channels = tf.unstack(new_image, axis=2)
......@@ -2480,6 +2508,115 @@ def rgb_to_gray(image):
return _rgb_to_grayscale(image)
def random_self_concat_image(
image, boxes, labels, label_weights, label_confidences=None,
multiclass_scores=None, concat_vertical_probability=0.1,
concat_horizontal_probability=0.1, seed=None,
preprocess_vars_cache=None):
"""Randomly concatenates the image with itself.
This function randomly concatenates the image with itself; the random
variables for vertical and horizontal concatenation are independent.
Afterwards, we adjust the old bounding boxes, and add new bounding boxes
for the new objects.
Args:
image: rank 3 float32 tensor containing 1 image -> [height, width, channels]
with pixel values varying between [0, 1].
boxes: rank 2 float32 tensor containing the bounding boxes -> [N, 4].
Boxes are in normalized form meaning their coordinates vary
between [0, 1].
Each row is in the form of [ymin, xmin, ymax, xmax].
labels: rank 1 int32 tensor containing the object classes.
label_weights: rank 1 float32 containing the label weights.
label_confidences: (optional) rank 1 float32 containing the label
confidences.
multiclass_scores: (optional) float32 tensor of shape
[num_instances, num_classes] representing the score for
each box for each class.
concat_vertical_probability: (optional) a tf.float32 scalar denoting the
probability of a vertical concatenation.
concat_horizontal_probability: (optional) a tf.float32 scalar denoting the
probability of a horizontal concatenation.
seed: random seed.
preprocess_vars_cache: PreprocessorCache object that records previously
performed augmentations. Updated in-place. If this
function is called multiple times with the same
non-null cache, it will perform deterministically.
Returns:
image: Image shape will be [new_height, new_width, channels].
boxes: boxes which is the same rank as input boxes. Boxes are in normalized
form.
if label_confidences is not None also returns:
maybe_concat_label_confidences: cropped label weights.
if multiclass_scores is not None also returns:
maybe_concat_multiclass_scores: cropped_multiclass_scores.
"""
concat_vertical = (tf.random_uniform([], seed=seed) <
concat_vertical_probability)
# Note the seed + 1 so we get some semblance of independence even with
# fixed seeds.
concat_horizontal = (tf.random_uniform([], seed=seed + 1 if seed else None)
< concat_horizontal_probability)
gen_func = lambda: (concat_vertical, concat_horizontal)
params = _get_or_create_preprocess_rand_vars(
gen_func, preprocessor_cache.PreprocessorCache.SELF_CONCAT_IMAGE,
preprocess_vars_cache)
concat_vertical, concat_horizontal = params
def _concat_image(image, boxes, labels, label_weights, axis):
"""Concats the image to itself on `axis`."""
output_images = tf.concat([image, image], axis=axis)
if axis == 0:
# Concat vertically, so need to reduce the y coordinates.
old_scaling = tf.to_float([0.5, 1.0, 0.5, 1.0])
new_translation = tf.to_float([0.5, 0.0, 0.5, 0.0])
elif axis == 1:
old_scaling = tf.to_float([1.0, 0.5, 1.0, 0.5])
new_translation = tf.to_float([0.0, 0.5, 0.0, 0.5])
old_boxes = old_scaling * boxes
new_boxes = old_boxes + new_translation
all_boxes = tf.concat([old_boxes, new_boxes], axis=0)
return [output_images, all_boxes, tf.tile(labels, [2]), tf.tile(
label_weights, [2])]
image, boxes, labels, label_weights = tf.cond(
concat_vertical,
lambda: _concat_image(image, boxes, labels, label_weights, axis=0),
lambda: [image, boxes, labels, label_weights],
strict=True)
outputs = tf.cond(
concat_horizontal,
lambda: _concat_image(image, boxes, labels, label_weights, axis=1),
lambda: [image, boxes, labels, label_weights],
strict=True)
if label_confidences is not None:
label_confidences = tf.cond(concat_vertical,
lambda: tf.tile(label_confidences, [2]),
lambda: label_confidences)
outputs.append(tf.cond(concat_horizontal,
lambda: tf.tile(label_confidences, [2]),
lambda: label_confidences))
if multiclass_scores is not None:
multiclass_scores = tf.cond(concat_vertical,
lambda: tf.tile(multiclass_scores, [2, 1]),
lambda: multiclass_scores)
outputs.append(tf.cond(concat_horizontal,
lambda: tf.tile(multiclass_scores, [2, 1]),
lambda: multiclass_scores))
return outputs
def ssd_random_crop(image,
boxes,
labels,
......@@ -2804,7 +2941,7 @@ def ssd_random_crop_fixed_aspect_ratio(
Boxes are in normalized form.
labels: new labels.
If mulitclass_scores, masks, or keypoints is not None, the function also
If multiclass_scores, masks, or keypoints is not None, the function also
returns:
multiclass_scores: rank 2 float32 tensor with shape
......@@ -3132,9 +3269,13 @@ def get_default_func_arg_map(include_label_weights=True,
groundtruth_label_weights,
groundtruth_label_confidences,
multiclass_scores,
groundtruth_instance_masks, groundtruth_keypoints),
groundtruth_instance_masks,
groundtruth_keypoints),
random_pad_image: (fields.InputDataFields.image,
fields.InputDataFields.groundtruth_boxes),
fields.InputDataFields.groundtruth_boxes,
groundtruth_keypoints),
random_absolute_pad_image: (fields.InputDataFields.image,
fields.InputDataFields.groundtruth_boxes),
random_crop_pad_image: (fields.InputDataFields.image,
fields.InputDataFields.groundtruth_boxes,
fields.InputDataFields.groundtruth_classes,
......@@ -3189,6 +3330,12 @@ def get_default_func_arg_map(include_label_weights=True,
subtract_channel_mean: (fields.InputDataFields.image,),
one_hot_encoding: (fields.InputDataFields.groundtruth_image_classes,),
rgb_to_gray: (fields.InputDataFields.image,),
random_self_concat_image: (fields.InputDataFields.image,
fields.InputDataFields.groundtruth_boxes,
fields.InputDataFields.groundtruth_classes,
groundtruth_label_weights,
groundtruth_label_confidences,
multiclass_scores),
ssd_random_crop: (fields.InputDataFields.image,
fields.InputDataFields.groundtruth_boxes,
fields.InputDataFields.groundtruth_classes,
......@@ -3302,6 +3449,7 @@ def preprocess(tensor_dict,
if (preprocess_vars_cache is not None and
'preprocess_vars_cache' in inspect.getargspec(func).args):
params['preprocess_vars_cache'] = preprocess_vars_cache
results = func(*args, **params)
if not isinstance(results, (list, tuple)):
results = (results,)
......
research/object_detection/core/preprocessor_cache.py
View file @ 05584085
......@@ -51,6 +51,7 @@ class PreprocessorCache(object):
ADD_BLACK_PATCH = 'add_black_patch'
SELECTOR = 'selector'
SELECTOR_TUPLES = 'selector_tuples'
SELF_CONCAT_IMAGE = 'self_concat_image'
SSD_CROP_SELECTOR_ID = 'ssd_crop_selector_id'
SSD_CROP_PAD_SELECTOR_ID = 'ssd_crop_pad_selector_id'
......@@ -60,7 +61,8 @@ class PreprocessorCache(object):
ADJUST_HUE, ADJUST_SATURATION, DISTORT_COLOR, STRICT_CROP_IMAGE,
CROP_IMAGE, PAD_IMAGE, CROP_TO_ASPECT_RATIO, RESIZE_METHOD,
PAD_TO_ASPECT_RATIO, BLACK_PATCHES, ADD_BLACK_PATCH, SELECTOR,
SELECTOR_TUPLES, SSD_CROP_SELECTOR_ID, SSD_CROP_PAD_SELECTOR_ID]
SELECTOR_TUPLES, SELF_CONCAT_IMAGE, SSD_CROP_SELECTOR_ID,
SSD_CROP_PAD_SELECTOR_ID]
def __init__(self):
self._history = defaultdict(dict)
......@@ -99,4 +101,3 @@ class PreprocessorCache(object):
if function_id not in self._VALID_FNS:
raise ValueError('Function id not recognized: %s.' % str(function_id))
self._history[function_id][key] = value
research/object_detection/core/preprocessor_test.py
View file @ 05584085
......@@ -2071,6 +2071,96 @@ class PreprocessorTest(tf.test.TestCase):
self.assertTrue(np.all((boxes_[:, 3] - boxes_[:, 1]) >= (
padded_boxes_[:, 3] - padded_boxes_[:, 1])))
def testRandomPadImageWithKeypoints(self):
preprocessing_options = [(preprocessor.normalize_image, {
'original_minval': 0,
'original_maxval': 255,
'target_minval': 0,
'target_maxval': 1
})]
images = self.createTestImages()
boxes = self.createTestBoxes()
labels = self.createTestLabels()
keypoints = self.createTestKeypoints()
tensor_dict = {
fields.InputDataFields.image: images,
fields.InputDataFields.groundtruth_boxes: boxes,
fields.InputDataFields.groundtruth_classes: labels,
fields.InputDataFields.groundtruth_keypoints: keypoints,
}
tensor_dict = preprocessor.preprocess(tensor_dict, preprocessing_options)
images = tensor_dict[fields.InputDataFields.image]
preprocessing_options = [(preprocessor.random_pad_image, {})]
padded_tensor_dict = preprocessor.preprocess(tensor_dict,
preprocessing_options)
padded_images = padded_tensor_dict[fields.InputDataFields.image]
padded_boxes = padded_tensor_dict[
fields.InputDataFields.groundtruth_boxes]
padded_keypoints = padded_tensor_dict[
fields.InputDataFields.groundtruth_keypoints]
boxes_shape = tf.shape(boxes)
padded_boxes_shape = tf.shape(padded_boxes)
keypoints_shape = tf.shape(keypoints)
padded_keypoints_shape = tf.shape(padded_keypoints)
images_shape = tf.shape(images)
padded_images_shape = tf.shape(padded_images)
with self.test_session() as sess:
(boxes_shape_, padded_boxes_shape_, keypoints_shape_,
padded_keypoints_shape_, images_shape_, padded_images_shape_, boxes_,
padded_boxes_, keypoints_, padded_keypoints_) = sess.run(
[boxes_shape, padded_boxes_shape, keypoints_shape,
padded_keypoints_shape, images_shape, padded_images_shape, boxes,
padded_boxes, keypoints, padded_keypoints])
self.assertAllEqual(boxes_shape_, padded_boxes_shape_)
self.assertAllEqual(keypoints_shape_, padded_keypoints_shape_)
self.assertTrue((images_shape_[1] >= padded_images_shape_[1] * 0.5).all)
self.assertTrue((images_shape_[2] >= padded_images_shape_[2] * 0.5).all)
self.assertTrue((images_shape_[1] <= padded_images_shape_[1]).all)
self.assertTrue((images_shape_[2] <= padded_images_shape_[2]).all)
self.assertTrue(np.all((boxes_[:, 2] - boxes_[:, 0]) >= (
padded_boxes_[:, 2] - padded_boxes_[:, 0])))
self.assertTrue(np.all((boxes_[:, 3] - boxes_[:, 1]) >= (
padded_boxes_[:, 3] - padded_boxes_[:, 1])))
self.assertTrue(np.all((keypoints_[1, :, 0] - keypoints_[0, :, 0]) >= (
padded_keypoints_[1, :, 0] - padded_keypoints_[0, :, 0])))
self.assertTrue(np.all((keypoints_[1, :, 1] - keypoints_[0, :, 1]) >= (
padded_keypoints_[1, :, 1] - padded_keypoints_[0, :, 1])))
def testRandomAbsolutePadImage(self):
images = self.createTestImages()
boxes = self.createTestBoxes()
labels = self.createTestLabels()
tensor_dict = {
fields.InputDataFields.image: tf.to_float(images),
fields.InputDataFields.groundtruth_boxes: boxes,
fields.InputDataFields.groundtruth_classes: labels,
}
height_padding = 10
width_padding = 20
preprocessing_options = [(preprocessor.random_absolute_pad_image, {
'max_height_padding': height_padding,
'max_width_padding': width_padding})]
padded_tensor_dict = preprocessor.preprocess(tensor_dict,
preprocessing_options)
original_shape = tf.shape(images)
final_shape = tf.shape(padded_tensor_dict[fields.InputDataFields.image])
with self.test_session() as sess:
_, height, width, _ = sess.run(original_shape)
for _ in range(100):
output_shape = sess.run(final_shape)
self.assertTrue(output_shape[1] >= height)
self.assertTrue(output_shape[1] < height + height_padding)
self.assertTrue(output_shape[2] >= width)
self.assertTrue(output_shape[2] < width + width_padding)
def testRandomCropPadImageWithCache(self):
preprocess_options = [(preprocessor.normalize_image, {
'original_minval': 0,
......@@ -2693,6 +2783,95 @@ class PreprocessorTest(tf.test.TestCase):
self.assertAllEqual([0, 1, 1, 0, 1], one_hot)
def testRandomSelfConcatImage(self):
tf.set_random_seed(24601)
images = self.createTestImages()
boxes = self.createTestBoxes()
labels = self.createTestLabels()
weights = self.createTestGroundtruthWeights()
confidences = weights
scores = self.createTestMultiClassScores()
tensor_dict = {
fields.InputDataFields.image: tf.to_float(images),
fields.InputDataFields.groundtruth_boxes: boxes,
fields.InputDataFields.groundtruth_classes: labels,
fields.InputDataFields.groundtruth_weights: weights,
fields.InputDataFields.groundtruth_confidences: confidences,
fields.InputDataFields.multiclass_scores: scores,
}
preprocessing_options = [(preprocessor.random_self_concat_image, {
'concat_vertical_probability': 0.5,
'concat_horizontal_probability': 0.5,
'seed': 24601,
})]
func_arg_map = preprocessor.get_default_func_arg_map(
True, True, True)
output_tensor_dict = preprocessor.preprocess(
tensor_dict, preprocessing_options, func_arg_map=func_arg_map)
final_shape = tf.shape(output_tensor_dict[fields.InputDataFields.image])[
1:3]
with self.test_session() as sess:
outputs = []
augment_height_only = False
augment_width_only = False
for _ in range(50):
original_boxes = sess.run(boxes)
shape, new_boxes, new_labels, new_confidences, new_scores = sess.run(
[final_shape,
output_tensor_dict[fields.InputDataFields.groundtruth_boxes],
output_tensor_dict[fields.InputDataFields.groundtruth_classes],
output_tensor_dict[fields.InputDataFields.groundtruth_confidences],
output_tensor_dict[fields.InputDataFields.multiclass_scores],
])
shape = np.array(shape)
outputs.append(shape)
if np.array_equal(shape, [8, 4]):
augment_height_only = True
self.assertEqual(
new_boxes.shape[0], 2 * boxes.shape[0])
self.assertAllClose(new_boxes[:2, :] * [2.0, 1.0, 2.0, 1.0],
original_boxes)
self.assertAllClose(
(new_boxes[2:, :] - [0.5, 0.0, 0.5, 0.0]) * [
2.0, 1.0, 2.0, 1.0],
original_boxes)
elif np.array_equal(shape, [4, 8]):
augment_width_only = True
self.assertEqual(
new_boxes.shape[0], 2 * boxes.shape[0])
self.assertAllClose(new_boxes[:2, :] * [1.0, 2.0, 1.0, 2.0],
original_boxes)
self.assertAllClose(
(new_boxes[2:, :] - [0.0, 0.5, 0.0, 0.5]) * [
1.0, 2.0, 1.0, 2.0],
original_boxes)
augmentation_factor = new_boxes.shape[0] / boxes.shape[0].value
self.assertEqual(new_labels.shape[0],
labels.shape[0].value * augmentation_factor)
self.assertEqual(new_confidences.shape[0],
confidences.shape[0].value * augmentation_factor)
self.assertEqual(new_scores.shape[0],
scores.shape[0].value * augmentation_factor)
max_height = max(x[0] for x in outputs)
max_width = max(x[1] for x in outputs)
self.assertEqual(max_height, 8)
self.assertEqual(max_width, 8)
self.assertEqual(augment_height_only, True)
self.assertEqual(augment_width_only, True)
def testSSDRandomCropWithCache(self):
preprocess_options = [
(preprocessor.normalize_image, {
......
research/object_detection/core/standard_fields.py
View file @ 05584085
......@@ -114,6 +114,9 @@ class DetectionResultFields(object):
detection_boundaries: contains an object boundary for each detection box.
detection_keypoints: contains detection keypoints for each detection box.
num_detections: number of detections in the batch.
raw_detection_boxes: contains decoded detection boxes without Non-Max
suppression.
raw_detection_scores: contains class score logits for raw detection boxes.
"""
source_id = 'source_id'
......@@ -125,6 +128,8 @@ class DetectionResultFields(object):
detection_boundaries = 'detection_boundaries'
detection_keypoints = 'detection_keypoints'
num_detections = 'num_detections'
raw_detection_boxes = 'raw_detection_boxes'
raw_detection_scores = 'raw_detection_scores'
class BoxListFields(object):
......
research/object_detection/core/target_assigner.py
View file @ 05584085
......@@ -166,6 +166,10 @@ class TargetAssigner(object):
num_gt_boxes = groundtruth_boxes.num_boxes()
groundtruth_weights = tf.ones([num_gt_boxes], dtype=tf.float32)
# set scores on the gt boxes
scores = 1 - groundtruth_labels[:, 0]
groundtruth_boxes.add_field(fields.BoxListFields.scores, scores)
with tf.control_dependencies(
[unmatched_shape_assert, labels_and_box_shapes_assert]):
match_quality_matrix = self._similarity_calc.compare(groundtruth_boxes,
......
research/object_detection/data/oid_v4_label_map.pbtxt 0 → 100644
View file @ 05584085
item {
name: "/m/011k07"
id: 1
display_name: "Tortoise"
}
item {
name: "/m/011q46kg"
id: 2
display_name: "Container"
}
item {
name: "/m/012074"
id: 3
display_name: "Magpie"
}
item {
name: "/m/0120dh"
id: 4
display_name: "Sea turtle"
}
item {
name: "/m/01226z"
id: 5
display_name: "Football"
}
item {
name: "/m/012n7d"
id: 6
display_name: "Ambulance"
}
item {
name: "/m/012w5l"
id: 7
display_name: "Ladder"
}
item {
name: "/m/012xff"
id: 8
display_name: "Toothbrush"
}
item {
name: "/m/012ysf"
id: 9
display_name: "Syringe"
}
item {
name: "/m/0130jx"
id: 10
display_name: "Sink"
}
item {
name: "/m/0138tl"
id: 11
display_name: "Toy"
}
item {
name: "/m/013y1f"
id: 12
display_name: "Organ"
}
item {
name: "/m/01432t"
id: 13
display_name: "Cassette deck"
}
item {
name: "/m/014j1m"
id: 14
display_name: "Apple"
}
item {
name: "/m/014sv8"
id: 15
display_name: "Human eye"
}
item {
name: "/m/014trl"
id: 16
display_name: "Cosmetics"
}
item {
name: "/m/014y4n"
id: 17
display_name: "Paddle"
}
item {
name: "/m/0152hh"
id: 18
display_name: "Snowman"
}
item {
name: "/m/01599"
id: 19
display_name: "Beer"
}
item {
name: "/m/01_5g"
id: 20
display_name: "Chopsticks"
}
item {
name: "/m/015h_t"
id: 21
display_name: "Human beard"
}
item {
name: "/m/015p6"
id: 22
display_name: "Bird"
}
item {
name: "/m/015qbp"
id: 23
display_name: "Parking meter"
}
item {
name: "/m/015qff"
id: 24
display_name: "Traffic light"
}
item {
name: "/m/015wgc"
id: 25
display_name: "Croissant"
}
item {
name: "/m/015x4r"
id: 26
display_name: "Cucumber"
}
item {
name: "/m/015x5n"
id: 27
display_name: "Radish"
}
item {
name: "/m/0162_1"
id: 28
display_name: "Towel"
}
item {
name: "/m/0167gd"
id: 29
display_name: "Doll"
}
item {
name: "/m/016m2d"
id: 30
display_name: "Skull"
}
item {
name: "/m/0174k2"
id: 31
display_name: "Washing machine"
}
item {
name: "/m/0174n1"
id: 32
display_name: "Glove"
}
item {
name: "/m/0175cv"
id: 33
display_name: "Tick"
}
item {
name: "/m/0176mf"
id: 34
display_name: "Belt"
}
item {
name: "/m/017ftj"
id: 35
display_name: "Sunglasses"
}
item {
name: "/m/018j2"
id: 36
display_name: "Banjo"
}
item {
name: "/m/018p4k"
id: 37
display_name: "Cart"
}
item {
name: "/m/018xm"
id: 38
display_name: "Ball"
}
item {
name: "/m/01940j"
id: 39
display_name: "Backpack"
}
item {
name: "/m/0199g"
id: 40
display_name: "Bicycle"
}
item {
name: "/m/019dx1"
id: 41
display_name: "Home appliance"
}
item {
name: "/m/019h78"
id: 42
display_name: "Centipede"
}
item {
name: "/m/019jd"
id: 43
display_name: "Boat"
}
item {
name: "/m/019w40"
id: 44
display_name: "Surfboard"
}
item {
name: "/m/01b638"
id: 45
display_name: "Boot"
}
item {
name: "/m/01b7fy"
id: 46
display_name: "Headphones"
}
item {
name: "/m/01b9xk"
id: 47
display_name: "Hot dog"
}
item {
name: "/m/01bfm9"
id: 48
display_name: "Shorts"
}
item {
name: "/m/01_bhs"
id: 49
display_name: "Fast food"
}
item {
name: "/m/01bjv"
id: 50
display_name: "Bus"
}
item {
name: "/m/01bl7v"
id: 51
display_name: "Boy"
}
item {
name: "/m/01bms0"
id: 52
display_name: "Screwdriver"
}
item {
name: "/m/01bqk0"
id: 53
display_name: "Bicycle wheel"
}
item {
name: "/m/01btn"
id: 54
display_name: "Barge"
}
item {
name: "/m/01c648"
id: 55
display_name: "Laptop"
}
item {
name: "/m/01cmb2"
id: 56
display_name: "Miniskirt"
}
item {
name: "/m/01d380"
id: 57
display_name: "Drill"
}
item {
name: "/m/01d40f"
id: 58
display_name: "Dress"
}
item {
name: "/m/01dws"
id: 59
display_name: "Bear"
}
item {
name: "/m/01dwsz"
id: 60
display_name: "Waffle"
}
item {
name: "/m/01dwwc"
id: 61
display_name: "Pancake"
}
item {
name: "/m/01dxs"
id: 62
display_name: "Brown bear"
}
item {
name: "/m/01dy8n"
id: 63
display_name: "Woodpecker"
}
item {
name: "/m/01f8m5"
id: 64
display_name: "Blue jay"
}
item {
name: "/m/01f91_"
id: 65
display_name: "Pretzel"
}
item {
name: "/m/01fb_0"
id: 66
display_name: "Bagel"
}
item {
name: "/m/01fdzj"
id: 67
display_name: "Tower"
}
item {
name: "/m/01fh4r"
id: 68
display_name: "Teapot"
}
item {
name: "/m/01g317"
id: 69
display_name: "Person"
}
item {
name: "/m/01g3x7"
id: 70
display_name: "Bow and arrow"
}
item {
name: "/m/01gkx_"
id: 71
display_name: "Swimwear"
}
item {
name: "/m/01gllr"
id: 72
display_name: "Beehive"
}
item {
name: "/m/01gmv2"
id: 73
display_name: "Brassiere"
}
item {
name: "/m/01h3n"
id: 74
display_name: "Bee"
}
item {
name: "/m/01h44"
id: 75
display_name: "Bat"
}
item {
name: "/m/01h8tj"
id: 76
display_name: "Starfish"
}
item {
name: "/m/01hrv5"
id: 77
display_name: "Popcorn"
}
item {
name: "/m/01j3zr"
id: 78
display_name: "Burrito"
}
item {
name: "/m/01j4z9"
id: 79
display_name: "Chainsaw"
}
item {
name: "/m/01j51"
id: 80
display_name: "Balloon"
}
item {
name: "/m/01j5ks"
id: 81
display_name: "Wrench"
}
item {
name: "/m/01j61q"
id: 82
display_name: "Tent"
}
item {
name: "/m/01jfm_"
id: 83
display_name: "Vehicle registration plate"
}
item {
name: "/m/01jfsr"
id: 84
display_name: "Lantern"
}
item {
name: "/m/01k6s3"
id: 85
display_name: "Toaster"
}
item {
name: "/m/01kb5b"
id: 86
display_name: "Flashlight"
}
item {
name: "/m/01knjb"
id: 87
display_name: "Billboard"
}
item {
name: "/m/01krhy"
id: 88
display_name: "Tiara"
}
item {
name: "/m/01lcw4"
id: 89
display_name: "Limousine"
}
item {
name: "/m/01llwg"
id: 90
display_name: "Necklace"
}
item {
name: "/m/01lrl"
id: 91
display_name: "Carnivore"
}
item {
name: "/m/01lsmm"
id: 92
display_name: "Scissors"
}
item {
name: "/m/01lynh"
id: 93
display_name: "Stairs"
}
item {
name: "/m/01m2v"
id: 94
display_name: "Computer keyboard"
}
item {
name: "/m/01m4t"
id: 95
display_name: "Printer"
}
item {
name: "/m/01mqdt"
id: 96
display_name: "Traffic sign"
}
item {
name: "/m/01mzpv"
id: 97
display_name: "Chair"
}
item {
name: "/m/01n4qj"
id: 98
display_name: "Shirt"
}
item {
name: "/m/01n5jq"
id: 99
display_name: "Poster"
}
item {
name: "/m/01nkt"
id: 100
display_name: "Cheese"
}
item {
name: "/m/01nq26"
id: 101
display_name: "Sock"
}
item {
name: "/m/01pns0"
id: 102
display_name: "Fire hydrant"
}
item {
name: "/m/01prls"
id: 103
display_name: "Land vehicle"
}
item {
name: "/m/01r546"
id: 104
display_name: "Earrings"
}
item {
name: "/m/01rkbr"
id: 105
display_name: "Tie"
}
item {
name: "/m/01rzcn"
id: 106
display_name: "Watercraft"
}
item {
name: "/m/01s105"
id: 107
display_name: "Cabinetry"
}
item {
name: "/m/01s55n"
id: 108
display_name: "Suitcase"
}
item {
name: "/m/01tcjp"
id: 109
display_name: "Muffin"
}
item {
name: "/m/01vbnl"
id: 110
display_name: "Bidet"
}
item {
name: "/m/01ww8y"
id: 111
display_name: "Snack"
}
item {
name: "/m/01x3jk"
id: 112
display_name: "Snowmobile"
}
item {
name: "/m/01x3z"
id: 113
display_name: "Clock"
}
item {
name: "/m/01xgg_"
id: 114
display_name: "Medical equipment"
}
item {
name: "/m/01xq0k1"
id: 115
display_name: "Cattle"
}
item {
name: "/m/01xqw"
id: 116
display_name: "Cello"
}
item {
name: "/m/01xs3r"
id: 117
display_name: "Jet ski"
}
item {
name: "/m/01x_v"
id: 118
display_name: "Camel"
}
item {
name: "/m/01xygc"
id: 119
display_name: "Coat"
}
item {
name: "/m/01xyhv"
id: 120
display_name: "Suit"
}
item {
name: "/m/01y9k5"
id: 121
display_name: "Desk"
}
item {
name: "/m/01yrx"
id: 122
display_name: "Cat"
}
item {
name: "/m/01yx86"
id: 123
display_name: "Bronze sculpture"
}
item {
name: "/m/01z1kdw"
id: 124
display_name: "Juice"
}
item {
name: "/m/02068x"
id: 125
display_name: "Gondola"
}
item {
name: "/m/020jm"
id: 126
display_name: "Beetle"
}
item {
name: "/m/020kz"
id: 127
display_name: "Cannon"
}
item {
name: "/m/020lf"
id: 128
display_name: "Computer mouse"
}
item {
name: "/m/021mn"
id: 129
display_name: "Cookie"
}
item {
name: "/m/021sj1"
id: 130
display_name: "Office building"
}
item {
name: "/m/0220r2"
id: 131
display_name: "Fountain"
}
item {
name: "/m/0242l"
id: 132
display_name: "Coin"
}
item {
name: "/m/024d2"
id: 133
display_name: "Calculator"
}
item {
name: "/m/024g6"
id: 134
display_name: "Cocktail"
}
item {
name: "/m/02522"
id: 135
display_name: "Computer monitor"
}
item {
name: "/m/025dyy"
id: 136
display_name: "Box"
}
item {
name: "/m/025fsf"
id: 137
display_name: "Stapler"
}
item {
name: "/m/025nd"
id: 138
display_name: "Christmas tree"
}
item {
name: "/m/025rp__"
id: 139
display_name: "Cowboy hat"
}
item {
name: "/m/0268lbt"
id: 140
display_name: "Hiking equipment"
}
item {
name: "/m/026qbn5"
id: 141
display_name: "Studio couch"
}
item {
name: "/m/026t6"
id: 142
display_name: "Drum"
}
item {
name: "/m/0270h"
id: 143
display_name: "Dessert"
}
item {
name: "/m/0271qf7"
id: 144
display_name: "Wine rack"
}
item {
name: "/m/0271t"
id: 145
display_name: "Drink"
}
item {
name: "/m/027pcv"
id: 146
display_name: "Zucchini"
}
item {
name: "/m/027rl48"
id: 147
display_name: "Ladle"
}
item {
name: "/m/0283dt1"
id: 148
display_name: "Human mouth"
}
item {
name: "/m/0284d"
id: 149
display_name: "Dairy"
}
item {
name: "/m/029b3"
id: 150
display_name: "Dice"
}
item {
name: "/m/029bxz"
id: 151
display_name: "Oven"
}
item {
name: "/m/029tx"
id: 152
display_name: "Dinosaur"
}
item {
name: "/m/02bm9n"
id: 153
display_name: "Ratchet"
}
item {
name: "/m/02crq1"
id: 154
display_name: "Couch"
}
item {
name: "/m/02ctlc"
id: 155
display_name: "Cricket ball"
}
item {
name: "/m/02cvgx"
id: 156
display_name: "Winter melon"
}
item {
name: "/m/02d1br"
id: 157
display_name: "Spatula"
}
item {
name: "/m/02d9qx"
id: 158
display_name: "Whiteboard"
}
item {
name: "/m/02ddwp"
id: 159
display_name: "Pencil sharpener"
}
item {
name: "/m/02dgv"
id: 160
display_name: "Door"
}
item {
name: "/m/02dl1y"
id: 161
display_name: "Hat"
}
item {
name: "/m/02f9f_"
id: 162
display_name: "Shower"
}
item {
name: "/m/02fh7f"
id: 163
display_name: "Eraser"
}
item {
name: "/m/02fq_6"
id: 164
display_name: "Fedora"
}
item {
name: "/m/02g30s"
id: 165
display_name: "Guacamole"
}
item {
name: "/m/02gzp"
id: 166
display_name: "Dagger"
}
item {
name: "/m/02h19r"
id: 167
display_name: "Scarf"
}
item {
name: "/m/02hj4"
id: 168
display_name: "Dolphin"
}
item {
name: "/m/02jfl0"
id: 169
display_name: "Sombrero"
}
item {
name: "/m/02jnhm"
id: 170
display_name: "Tin can"
}
item {
name: "/m/02jvh9"
id: 171
display_name: "Mug"
}
item {
name: "/m/02jz0l"
id: 172
display_name: "Tap"
}
item {
name: "/m/02l8p9"
id: 173
display_name: "Harbor seal"
}
item {
name: "/m/02lbcq"
id: 174
display_name: "Stretcher"
}
item {
name: "/m/02mqfb"
id: 175
display_name: "Can opener"
}
item {
name: "/m/02_n6y"
id: 176
display_name: "Goggles"
}
item {
name: "/m/02p0tk3"
id: 177
display_name: "Human body"
}
item {
name: "/m/02p3w7d"
id: 178
display_name: "Roller skates"
}
item {
name: "/m/02p5f1q"
id: 179
display_name: "Coffee cup"
}
item {
name: "/m/02pdsw"
id: 180
display_name: "Cutting board"
}
item {
name: "/m/02pjr4"
id: 181
display_name: "Blender"
}
item {
name: "/m/02pkr5"
id: 182
display_name: "Plumbing fixture"
}
item {
name: "/m/02pv19"
id: 183
display_name: "Stop sign"
}
item {
name: "/m/02rdsp"
id: 184
display_name: "Office supplies"
}
item {
name: "/m/02rgn06"
id: 185
display_name: "Volleyball"
}
item {
name: "/m/02s195"
id: 186
display_name: "Vase"
}
item {
name: "/m/02tsc9"
id: 187
display_name: "Slow cooker"
}
item {
name: "/m/02vkqh8"
id: 188
display_name: "Wardrobe"
}
item {
name: "/m/02vqfm"
id: 189
display_name: "Coffee"
}
item {
name: "/m/02vwcm"
id: 190
display_name: "Whisk"
}
item {
name: "/m/02w3r3"
id: 191
display_name: "Paper towel"
}
item {
name: "/m/02w3_ws"
id: 192
display_name: "Personal care"
}
item {
name: "/m/02wbm"
id: 193
display_name: "Food"
}
item {
name: "/m/02wbtzl"
id: 194
display_name: "Sun hat"
}
item {
name: "/m/02wg_p"
id: 195
display_name: "Tree house"
}
item {
name: "/m/02wmf"
id: 196
display_name: "Flying disc"
}
item {
name: "/m/02wv6h6"
id: 197
display_name: "Skirt"
}
item {
name: "/m/02wv84t"
id: 198
display_name: "Gas stove"
}
item {
name: "/m/02x8cch"
id: 199
display_name: "Salt and pepper shakers"
}
item {
name: "/m/02x984l"
id: 200
display_name: "Mechanical fan"
}
item {
name: "/m/02xb7qb"
id: 201
display_name: "Face powder"
}
item {
name: "/m/02xqq"
id: 202
display_name: "Fax"
}
item {
name: "/m/02xwb"
id: 203
display_name: "Fruit"
}
item {
name: "/m/02y6n"
id: 204
display_name: "French fries"
}
item {
name: "/m/02z51p"
id: 205
display_name: "Nightstand"
}
item {
name: "/m/02zn6n"
id: 206
display_name: "Barrel"
}
item {
name: "/m/02zt3"
id: 207
display_name: "Kite"
}
item {
name: "/m/02zvsm"
id: 208
display_name: "Tart"
}
item {
name: "/m/030610"
id: 209
display_name: "Treadmill"
}
item {
name: "/m/0306r"
id: 210
display_name: "Fox"
}
item {
name: "/m/03120"
id: 211
display_name: "Flag"
}
item {
name: "/m/0319l"
id: 212
display_name: "Horn"
}
item {
name: "/m/031b6r"
id: 213
display_name: "Window blind"
}
item {
name: "/m/031n1"
id: 214
display_name: "Human foot"
}
item {
name: "/m/0323sq"
id: 215
display_name: "Golf cart"
}
item {
name: "/m/032b3c"
id: 216
display_name: "Jacket"
}
item {
name: "/m/033cnk"
id: 217
display_name: "Egg"
}
item {
name: "/m/033rq4"
id: 218
display_name: "Street light"
}
item {
name: "/m/0342h"
id: 219
display_name: "Guitar"
}
item {
name: "/m/034c16"
id: 220
display_name: "Pillow"
}
item {
name: "/m/035r7c"
id: 221
display_name: "Human leg"
}
item {
name: "/m/035vxb"
id: 222
display_name: "Isopod"
}
item {
name: "/m/0388q"
id: 223
display_name: "Grape"
}
item {
name: "/m/039xj_"
id: 224
display_name: "Human ear"
}
item {
name: "/m/03bbps"
id: 225
display_name: "Power plugs and sockets"
}
item {
name: "/m/03bj1"
id: 226
display_name: "Panda"
}
item {
name: "/m/03bk1"
id: 227
display_name: "Giraffe"
}
item {
name: "/m/03bt1vf"
id: 228
display_name: "Woman"
}
item {
name: "/m/03c7gz"
id: 229
display_name: "Door handle"
}
item {
name: "/m/03d443"
id: 230
display_name: "Rhinoceros"
}
item {
name: "/m/03dnzn"
id: 231
display_name: "Bathtub"
}
item {
name: "/m/03fj2"
id: 232
display_name: "Goldfish"
}
item {
name: "/m/03fp41"
id: 233
display_name: "Houseplant"
}
item {
name: "/m/03fwl"
id: 234
display_name: "Goat"
}
item {
name: "/m/03g8mr"
id: 235
display_name: "Baseball bat"
}
item {
name: "/m/03grzl"
id: 236
display_name: "Baseball glove"
}
item {
name: "/m/03hj559"
id: 237
display_name: "Mixing bowl"
}
item {
name: "/m/03hl4l9"
id: 238
display_name: "Marine invertebrates"
}
item {
name: "/m/03hlz0c"
id: 239
display_name: "Kitchen utensil"
}
item {
name: "/m/03jbxj"
id: 240
display_name: "Light switch"
}
item {
name: "/m/03jm5"
id: 241
display_name: "House"
}
item {
name: "/m/03k3r"
id: 242
display_name: "Horse"
}
item {
name: "/m/03kt2w"
id: 243
display_name: "Stationary bicycle"
}
item {
name: "/m/03l9g"
id: 244
display_name: "Hammer"
}
item {
name: "/m/03ldnb"
id: 245
display_name: "Ceiling fan"
}
item {
name: "/m/03m3pdh"
id: 246
display_name: "Sofa bed"
}
item {
name: "/m/03m3vtv"
id: 247
display_name: "Adhesive tape"
}
item {
name: "/m/03m5k"
id: 248
display_name: "Harp"
}
item {
name: "/m/03nfch"
id: 249
display_name: "Sandal"
}
item {
name: "/m/03p3bw"
id: 250
display_name: "Bicycle helmet"
}
item {
name: "/m/03q5c7"
id: 251
display_name: "Saucer"
}
item {
name: "/m/03q5t"
id: 252
display_name: "Harpsichord"
}
item {
name: "/m/03q69"
id: 253
display_name: "Human hair"
}
item {
name: "/m/03qhv5"
id: 254
display_name: "Heater"
}
item {
name: "/m/03qjg"
id: 255
display_name: "Harmonica"
}
item {
name: "/m/03qrc"
id: 256
display_name: "Hamster"
}
item {
name: "/m/03rszm"
id: 257
display_name: "Curtain"
}
item {
name: "/m/03ssj5"
id: 258
display_name: "Bed"
}
item {
name: "/m/03s_tn"
id: 259
display_name: "Kettle"
}
item {
name: "/m/03tw93"
id: 260
display_name: "Fireplace"
}
item {
name: "/m/03txqz"
id: 261
display_name: "Scale"
}
item {
name: "/m/03v5tg"
id: 262
display_name: "Drinking straw"
}
item {
name: "/m/03vt0"
id: 263
display_name: "Insect"
}
item {
name: "/m/03wvsk"
id: 264
display_name: "Hair dryer"
}
item {
name: "/m/03_wxk"
id: 265
display_name: "Kitchenware"
}
item {
name: "/m/03wym"
id: 266
display_name: "Indoor rower"
}
item {
name: "/m/03xxp"
id: 267
display_name: "Invertebrate"
}
item {
name: "/m/03y6mg"
id: 268
display_name: "Food processor"
}
item {
name: "/m/03__z0"
id: 269
display_name: "Bookcase"
}
item {
name: "/m/040b_t"
id: 270
display_name: "Refrigerator"
}
item {
name: "/m/04169hn"
id: 271
display_name: "Wood-burning stove"
}
item {
name: "/m/0420v5"
id: 272
display_name: "Punching bag"
}
item {
name: "/m/043nyj"
id: 273
display_name: "Common fig"
}
item {
name: "/m/0440zs"
id: 274
display_name: "Cocktail shaker"
}
item {
name: "/m/0449p"
id: 275
display_name: "Jaguar"
}
item {
name: "/m/044r5d"
id: 276
display_name: "Golf ball"
}
item {
name: "/m/0463sg"
id: 277
display_name: "Fashion accessory"
}
item {
name: "/m/046dlr"
id: 278
display_name: "Alarm clock"
}
item {
name: "/m/047j0r"
id: 279
display_name: "Filing cabinet"
}
item {
name: "/m/047v4b"
id: 280
display_name: "Artichoke"
}
item {
name: "/m/04bcr3"
id: 281
display_name: "Table"
}
item {
name: "/m/04brg2"
id: 282
display_name: "Tableware"
}
item {
name: "/m/04c0y"
id: 283
display_name: "Kangaroo"
}
item {
name: "/m/04cp_"
id: 284
display_name: "Koala"
}
item {
name: "/m/04ctx"
id: 285
display_name: "Knife"
}
item {
name: "/m/04dr76w"
id: 286
display_name: "Bottle"
}
item {
name: "/m/04f5ws"
id: 287
display_name: "Bottle opener"
}
item {
name: "/m/04g2r"
id: 288
display_name: "Lynx"
}
item {
name: "/m/04gth"
id: 289
display_name: "Lavender"
}
item {
name: "/m/04h7h"
id: 290
display_name: "Lighthouse"
}
item {
name: "/m/04h8sr"
id: 291
display_name: "Dumbbell"
}
item {
name: "/m/04hgtk"
id: 292
display_name: "Human head"
}
item {
name: "/m/04kkgm"
id: 293
display_name: "Bowl"
}
item {
name: "/m/04lvq_"
id: 294
display_name: "Humidifier"
}
item {
name: "/m/04m6gz"
id: 295
display_name: "Porch"
}
item {
name: "/m/04m9y"
id: 296
display_name: "Lizard"
}
item {
name: "/m/04p0qw"
id: 297
display_name: "Billiard table"
}
item {
name: "/m/04rky"
id: 298
display_name: "Mammal"
}
item {
name: "/m/04rmv"
id: 299
display_name: "Mouse"
}
item {
name: "/m/04_sv"
id: 300
display_name: "Motorcycle"
}
item {
name: "/m/04szw"
id: 301
display_name: "Musical instrument"
}
item {
name: "/m/04tn4x"
id: 302
display_name: "Swim cap"
}
item {
name: "/m/04v6l4"
id: 303
display_name: "Frying pan"
}
item {
name: "/m/04vv5k"
id: 304
display_name: "Snowplow"
}
item {
name: "/m/04y4h8h"
id: 305
display_name: "Bathroom cabinet"
}
item {
name: "/m/04ylt"
id: 306
display_name: "Missile"
}
item {
name: "/m/04yqq2"
id: 307
display_name: "Bust"
}
item {
name: "/m/04yx4"
id: 308
display_name: "Man"
}
item {
name: "/m/04z4wx"
id: 309
display_name: "Waffle iron"
}
item {
name: "/m/04zpv"
id: 310
display_name: "Milk"
}
item {
name: "/m/04zwwv"
id: 311
display_name: "Ring binder"
}
item {
name: "/m/050gv4"
id: 312
display_name: "Plate"
}
item {
name: "/m/050k8"
id: 313
display_name: "Mobile phone"
}
item {
name: "/m/052lwg6"
id: 314
display_name: "Baked goods"
}
item {
name: "/m/052sf"
id: 315
display_name: "Mushroom"
}
item {
name: "/m/05441v"
id: 316
display_name: "Crutch"
}
item {
name: "/m/054fyh"
id: 317
display_name: "Pitcher"
}
item {
name: "/m/054_l"
id: 318
display_name: "Mirror"
}
item {
name: "/m/054xkw"
id: 319
display_name: "Lifejacket"
}
item {
name: "/m/05_5p_0"
id: 320
display_name: "Table tennis racket"
}
item {
name: "/m/05676x"
id: 321
display_name: "Pencil case"
}
item {
name: "/m/057cc"
id: 322
display_name: "Musical keyboard"
}
item {
name: "/m/057p5t"
id: 323
display_name: "Scoreboard"
}
item {
name: "/m/0584n8"
id: 324
display_name: "Briefcase"
}
item {
name: "/m/058qzx"
id: 325
display_name: "Kitchen knife"
}
item {
name: "/m/05bm6"
id: 326
display_name: "Nail"
}
item {
name: "/m/05ctyq"
id: 327
display_name: "Tennis ball"
}
item {
name: "/m/05gqfk"
id: 328
display_name: "Plastic bag"
}
item {
name: "/m/05kms"
id: 329
display_name: "Oboe"
}
item {
name: "/m/05kyg_"
id: 330
display_name: "Chest of drawers"
}
item {
name: "/m/05n4y"
id: 331
display_name: "Ostrich"
}
item {
name: "/m/05r5c"
id: 332
display_name: "Piano"
}
item {
name: "/m/05r655"
id: 333
display_name: "Girl"
}
item {
name: "/m/05s2s"
id: 334
display_name: "Plant"
}
item {
name: "/m/05vtc"
id: 335
display_name: "Potato"
}
item {
name: "/m/05w9t9"
id: 336
display_name: "Hair spray"
}
item {
name: "/m/05y5lj"
id: 337
display_name: "Sports equipment"
}
item {
name: "/m/05z55"
id: 338
display_name: "Pasta"
}
item {
name: "/m/05z6w"
id: 339
display_name: "Penguin"
}
item {
name: "/m/05zsy"
id: 340
display_name: "Pumpkin"
}
item {
name: "/m/061_f"
id: 341
display_name: "Pear"
}
item {
name: "/m/061hd_"
id: 342
display_name: "Infant bed"
}
item {
name: "/m/0633h"
id: 343
display_name: "Polar bear"
}
item {
name: "/m/063rgb"
id: 344
display_name: "Mixer"
}
item {
name: "/m/0642b4"
id: 345
display_name: "Cupboard"
}
item {
name: "/m/065h6l"
id: 346
display_name: "Jacuzzi"
}
item {
name: "/m/0663v"
id: 347
display_name: "Pizza"
}
item {
name: "/m/06_72j"
id: 348
display_name: "Digital clock"
}
item {
name: "/m/068zj"
id: 349
display_name: "Pig"
}
item {
name: "/m/06bt6"
id: 350
display_name: "Reptile"
}
item {
name: "/m/06c54"
id: 351
display_name: "Rifle"
}
item {
name: "/m/06c7f7"
id: 352
display_name: "Lipstick"
}
item {
name: "/m/06_fw"
id: 353
display_name: "Skateboard"
}
item {
name: "/m/06j2d"
id: 354
display_name: "Raven"
}
item {
name: "/m/06k2mb"
id: 355
display_name: "High heels"
}
item {
name: "/m/06l9r"
id: 356
display_name: "Red panda"
}
item {
name: "/m/06m11"
id: 357
display_name: "Rose"
}
item {
name: "/m/06mf6"
id: 358
display_name: "Rabbit"
}
item {
name: "/m/06msq"
id: 359
display_name: "Sculpture"
}
item {
name: "/m/06ncr"
id: 360
display_name: "Saxophone"
}
item {
name: "/m/06nrc"
id: 361
display_name: "Shotgun"
}
item {
name: "/m/06nwz"
id: 362
display_name: "Seafood"
}
item {
name: "/m/06pcq"
id: 363
display_name: "Submarine sandwich"
}
item {
name: "/m/06__v"
id: 364
display_name: "Snowboard"
}
item {
name: "/m/06y5r"
id: 365
display_name: "Sword"
}
item {
name: "/m/06z37_"
id: 366
display_name: "Picture frame"
}
item {
name: "/m/07030"
id: 367
display_name: "Sushi"
}
item {
name: "/m/0703r8"
id: 368
display_name: "Loveseat"
}
item {
name: "/m/071p9"
id: 369
display_name: "Ski"
}
item {
name: "/m/071qp"
id: 370
display_name: "Squirrel"
}
item {
name: "/m/073bxn"
id: 371
display_name: "Tripod"
}
item {
name: "/m/073g6"
id: 372
display_name: "Stethoscope"
}
item {
name: "/m/074d1"
id: 373
display_name: "Submarine"
}
item {
name: "/m/0755b"
id: 374
display_name: "Scorpion"
}
item {
name: "/m/076bq"
id: 375
display_name: "Segway"
}
item {
name: "/m/076lb9"
id: 376
display_name: "Training bench"
}
item {
name: "/m/078jl"
id: 377
display_name: "Snake"
}
item {
name: "/m/078n6m"
id: 378
display_name: "Coffee table"
}
item {
name: "/m/079cl"
id: 379
display_name: "Skyscraper"
}
item {
name: "/m/07bgp"
id: 380
display_name: "Sheep"
}
item {
name: "/m/07c52"
id: 381
display_name: "Television"
}
item {
name: "/m/07c6l"
id: 382
display_name: "Trombone"
}
item {
name: "/m/07clx"
id: 383
display_name: "Tea"
}
item {
name: "/m/07cmd"
id: 384
display_name: "Tank"
}
item {
name: "/m/07crc"
id: 385
display_name: "Taco"
}
item {
name: "/m/07cx4"
id: 386
display_name: "Telephone"
}
item {
name: "/m/07dd4"
id: 387
display_name: "Torch"
}
item {
name: "/m/07dm6"
id: 388
display_name: "Tiger"
}
item {
name: "/m/07fbm7"
id: 389
display_name: "Strawberry"
}
item {
name: "/m/07gql"
id: 390
display_name: "Trumpet"
}
item {
name: "/m/07j7r"
id: 391
display_name: "Tree"
}
item {
name: "/m/07j87"
id: 392
display_name: "Tomato"
}
item {
name: "/m/07jdr"
id: 393
display_name: "Train"
}
item {
name: "/m/07k1x"
id: 394
display_name: "Tool"
}
item {
name: "/m/07kng9"
id: 395
display_name: "Picnic basket"
}
item {
name: "/m/07mcwg"
id: 396
display_name: "Cooking spray"
}
item {
name: "/m/07mhn"
id: 397
display_name: "Trousers"
}
item {
name: "/m/07pj7bq"
id: 398
display_name: "Bowling equipment"
}
item {
name: "/m/07qxg_"
id: 399
display_name: "Football helmet"
}
item {
name: "/m/07r04"
id: 400
display_name: "Truck"
}
item {
name: "/m/07v9_z"
id: 401
display_name: "Measuring cup"
}
item {
name: "/m/07xyvk"
id: 402
display_name: "Coffeemaker"
}
item {
name: "/m/07y_7"
id: 403
display_name: "Violin"
}
item {
name: "/m/07yv9"
id: 404
display_name: "Vehicle"
}
item {
name: "/m/080hkjn"
id: 405
display_name: "Handbag"
}
item {
name: "/m/080n7g"
id: 406
display_name: "Paper cutter"
}
item {
name: "/m/081qc"
id: 407
display_name: "Wine"
}
item {
name: "/m/083kb"
id: 408
display_name: "Weapon"
}
item {
name: "/m/083wq"
id: 409
display_name: "Wheel"
}
item {
name: "/m/084hf"
id: 410
display_name: "Worm"
}
item {
name: "/m/084rd"
id: 411
display_name: "Wok"
}
item {
name: "/m/084zz"
id: 412
display_name: "Whale"
}
item {
name: "/m/0898b"
id: 413
display_name: "Zebra"
}
item {
name: "/m/08dz3q"
id: 414
display_name: "Auto part"
}
item {
name: "/m/08hvt4"
id: 415
display_name: "Jug"
}
item {
name: "/m/08ks85"
id: 416
display_name: "Pizza cutter"
}
item {
name: "/m/08p92x"
id: 417
display_name: "Cream"
}
item {
name: "/m/08pbxl"
id: 418
display_name: "Monkey"
}
item {
name: "/m/096mb"
id: 419
display_name: "Lion"
}
item {
name: "/m/09728"
id: 420
display_name: "Bread"
}
item {
name: "/m/099ssp"
id: 421
display_name: "Platter"
}
item {
name: "/m/09b5t"
id: 422
display_name: "Chicken"
}
item {
name: "/m/09csl"
id: 423
display_name: "Eagle"
}
item {
name: "/m/09ct_"
id: 424
display_name: "Helicopter"
}
item {
name: "/m/09d5_"
id: 425
display_name: "Owl"
}
item {
name: "/m/09ddx"
id: 426
display_name: "Duck"
}
item {
name: "/m/09dzg"
id: 427
display_name: "Turtle"
}
item {
name: "/m/09f20"
id: 428
display_name: "Hippopotamus"
}
item {
name: "/m/09f_2"
id: 429
display_name: "Crocodile"
}
item {
name: "/m/09g1w"
id: 430
display_name: "Toilet"
}
item {
name: "/m/09gtd"
id: 431
display_name: "Toilet paper"
}
item {
name: "/m/09gys"
id: 432
display_name: "Squid"
}
item {
name: "/m/09j2d"
id: 433
display_name: "Clothing"
}
item {
name: "/m/09j5n"
id: 434
display_name: "Footwear"
}
item {
name: "/m/09k_b"
id: 435
display_name: "Lemon"
}
item {
name: "/m/09kmb"
id: 436
display_name: "Spider"
}
item {
name: "/m/09kx5"
id: 437
display_name: "Deer"
}
item {
name: "/m/09ld4"
id: 438
display_name: "Frog"
}
item {
name: "/m/09qck"
id: 439
display_name: "Banana"
}
item {
name: "/m/09rvcxw"
id: 440
display_name: "Rocket"
}
item {
name: "/m/09tvcd"
id: 441
display_name: "Wine glass"
}
item {
name: "/m/0b3fp9"
id: 442
display_name: "Countertop"
}
item {
name: "/m/0bh9flk"
id: 443
display_name: "Tablet computer"
}
item {
name: "/m/0bjyj5"
id: 444
display_name: "Waste container"
}
item {
name: "/m/0b_rs"
id: 445
display_name: "Swimming pool"
}
item {
name: "/m/0bt9lr"
id: 446
display_name: "Dog"
}
item {
name: "/m/0bt_c3"
id: 447
display_name: "Book"
}
item {
name: "/m/0bwd_0j"
id: 448
display_name: "Elephant"
}
item {
name: "/m/0by6g"
id: 449
display_name: "Shark"
}
item {
name: "/m/0c06p"
id: 450
display_name: "Candle"
}
item {
name: "/m/0c29q"
id: 451
display_name: "Leopard"
}
item {
name: "/m/0c2jj"
id: 452
display_name: "Axe"
}
item {
name: "/m/0c3m8g"
id: 453
display_name: "Hand dryer"
}
item {
name: "/m/0c3mkw"
id: 454
display_name: "Soap dispenser"
}
item {
name: "/m/0c568"
id: 455
display_name: "Porcupine"
}
item {
name: "/m/0c9ph5"
id: 456
display_name: "Flower"
}
item {
name: "/m/0ccs93"
id: 457
display_name: "Canary"
}
item {
name: "/m/0cd4d"
id: 458
display_name: "Cheetah"
}
item {
name: "/m/0cdl1"
id: 459
display_name: "Palm tree"
}
item {
name: "/m/0cdn1"
id: 460
display_name: "Hamburger"
}
item {
name: "/m/0cffdh"
id: 461
display_name: "Maple"
}
item {
name: "/m/0cgh4"
id: 462
display_name: "Building"
}
item {
name: "/m/0ch_cf"
id: 463
display_name: "Fish"
}
item {
name: "/m/0cjq5"
id: 464
display_name: "Lobster"
}
item {
name: "/m/0cjs7"
id: 465
display_name: "Asparagus"
}
item {
name: "/m/0c_jw"
id: 466
display_name: "Furniture"
}
item {
name: "/m/0cl4p"
id: 467
display_name: "Hedgehog"
}
item {
name: "/m/0cmf2"
id: 468
display_name: "Airplane"
}
item {
name: "/m/0cmx8"
id: 469
display_name: "Spoon"
}
item {
name: "/m/0cn6p"
id: 470
display_name: "Otter"
}
item {
name: "/m/0cnyhnx"
id: 471
display_name: "Bull"
}
item {
name: "/m/0_cp5"
id: 472
display_name: "Oyster"
}
item {
name: "/m/0cqn2"
id: 473
display_name: "Horizontal bar"
}
item {
name: "/m/0crjs"
id: 474
display_name: "Convenience store"
}
item {
name: "/m/0ct4f"
id: 475
display_name: "Bomb"
}
item {
name: "/m/0cvnqh"
id: 476
display_name: "Bench"
}
item {
name: "/m/0cxn2"
id: 477
display_name: "Ice cream"
}
item {
name: "/m/0cydv"
id: 478
display_name: "Caterpillar"
}
item {
name: "/m/0cyf8"
id: 479
display_name: "Butterfly"
}
item {
name: "/m/0cyfs"
id: 480
display_name: "Parachute"
}
item {
name: "/m/0cyhj_"
id: 481
display_name: "Orange"
}
item {
name: "/m/0czz2"
id: 482
display_name: "Antelope"
}
item {
name: "/m/0d20w4"
id: 483
display_name: "Beaker"
}
item {
name: "/m/0d_2m"
id: 484
display_name: "Moths and butterflies"
}
item {
name: "/m/0d4v4"
id: 485
display_name: "Window"
}
item {
name: "/m/0d4w1"
id: 486
display_name: "Closet"
}
item {
name: "/m/0d5gx"
id: 487
display_name: "Castle"
}
item {
name: "/m/0d8zb"
id: 488
display_name: "Jellyfish"
}
item {
name: "/m/0dbvp"
id: 489
display_name: "Goose"
}
item {
name: "/m/0dbzx"
id: 490
display_name: "Mule"
}
item {
name: "/m/0dftk"
id: 491
display_name: "Swan"
}
item {
name: "/m/0dj6p"
id: 492
display_name: "Peach"
}
item {
name: "/m/0djtd"
id: 493
display_name: "Coconut"
}
item {
name: "/m/0dkzw"
id: 494
display_name: "Seat belt"
}
item {
name: "/m/0dq75"
id: 495
display_name: "Raccoon"
}
item {
name: "/m/0_dqb"
id: 496
display_name: "Chisel"
}
item {
name: "/m/0dt3t"
id: 497
display_name: "Fork"
}
item {
name: "/m/0dtln"
id: 498
display_name: "Lamp"
}
item {
name: "/m/0dv5r"
id: 499
display_name: "Camera"
}
item {
name: "/m/0dv77"
id: 500
display_name: "Squash"
}
item {
name: "/m/0dv9c"
id: 501
display_name: "Racket"
}
item {
name: "/m/0dzct"
id: 502
display_name: "Human face"
}
item {
name: "/m/0dzf4"
id: 503
display_name: "Human arm"
}
item {
name: "/m/0f4s2w"
id: 504
display_name: "Vegetable"
}
item {
name: "/m/0f571"
id: 505
display_name: "Diaper"
}
item {
name: "/m/0f6nr"
id: 506
display_name: "Unicycle"
}
item {
name: "/m/0f6wt"
id: 507
display_name: "Falcon"
}
item {
name: "/m/0f8s22"
id: 508
display_name: "Chime"
}
item {
name: "/m/0f9_l"
id: 509
display_name: "Snail"
}
item {
name: "/m/0fbdv"
id: 510
display_name: "Shellfish"
}
item {
name: "/m/0fbw6"
id: 511
display_name: "Cabbage"
}
item {
name: "/m/0fj52s"
id: 512
display_name: "Carrot"
}
item {
name: "/m/0fldg"
id: 513
display_name: "Mango"
}
item {
name: "/m/0fly7"
id: 514
display_name: "Jeans"
}
item {
name: "/m/0fm3zh"
id: 515
display_name: "Flowerpot"
}
item {
name: "/m/0fp6w"
id: 516
display_name: "Pineapple"
}
item {
name: "/m/0fqfqc"
id: 517
display_name: "Drawer"
}
item {
name: "/m/0fqt361"
id: 518
display_name: "Stool"
}
item {
name: "/m/0frqm"
id: 519
display_name: "Envelope"
}
item {
name: "/m/0fszt"
id: 520
display_name: "Cake"
}
item {
name: "/m/0ft9s"
id: 521
display_name: "Dragonfly"
}
item {
name: "/m/0ftb8"
id: 522
display_name: "Sunflower"
}
item {
name: "/m/0fx9l"
id: 523
display_name: "Microwave oven"
}
item {
name: "/m/0fz0h"
id: 524
display_name: "Honeycomb"
}
item {
name: "/m/0gd2v"
id: 525
display_name: "Marine mammal"
}
item {
name: "/m/0gd36"
id: 526
display_name: "Sea lion"
}
item {
name: "/m/0gj37"
id: 527
display_name: "Ladybug"
}
item {
name: "/m/0gjbg72"
id: 528
display_name: "Shelf"
}
item {
name: "/m/0gjkl"
id: 529
display_name: "Watch"
}
item {
name: "/m/0gm28"
id: 530
display_name: "Candy"
}
item {
name: "/m/0grw1"
id: 531
display_name: "Salad"
}
item {
name: "/m/0gv1x"
id: 532
display_name: "Parrot"
}
item {
name: "/m/0gxl3"
id: 533
display_name: "Handgun"
}
item {
name: "/m/0h23m"
id: 534
display_name: "Sparrow"
}
item {
name: "/m/0h2r6"
id: 535
display_name: "Van"
}
item {
name: "/m/0h8jyh6"
id: 536
display_name: "Grinder"
}
item {
name: "/m/0h8kx63"
id: 537
display_name: "Spice rack"
}
item {
name: "/m/0h8l4fh"
id: 538
display_name: "Light bulb"
}
item {
name: "/m/0h8lkj8"
id: 539
display_name: "Corded phone"
}
item {
name: "/m/0h8mhzd"
id: 540
display_name: "Sports uniform"
}
item {
name: "/m/0h8my_4"
id: 541
display_name: "Tennis racket"
}
item {
name: "/m/0h8mzrc"
id: 542
display_name: "Wall clock"
}
item {
name: "/m/0h8n27j"
id: 543
display_name: "Serving tray"
}
item {
name: "/m/0h8n5zk"
id: 544
display_name: "Kitchen & dining room table"
}
item {
name: "/m/0h8n6f9"
id: 545
display_name: "Dog bed"
}
item {
name: "/m/0h8n6ft"
id: 546
display_name: "Cake stand"
}
item {
name: "/m/0h8nm9j"
id: 547
display_name: "Cat furniture"
}
item {
name: "/m/0h8nr_l"
id: 548
display_name: "Bathroom accessory"
}
item {
name: "/m/0h8nsvg"
id: 549
display_name: "Facial tissue holder"
}
item {
name: "/m/0h8ntjv"
id: 550
display_name: "Pressure cooker"
}
item {
name: "/m/0h99cwc"
id: 551
display_name: "Kitchen appliance"
}
item {
name: "/m/0h9mv"
id: 552
display_name: "Tire"
}
item {
name: "/m/0hdln"
id: 553
display_name: "Ruler"
}
item {
name: "/m/0hf58v5"
id: 554
display_name: "Luggage and bags"
}
item {
name: "/m/0hg7b"
id: 555
display_name: "Microphone"
}
item {
name: "/m/0hkxq"
id: 556
display_name: "Broccoli"
}
item {
name: "/m/0hnnb"
id: 557
display_name: "Umbrella"
}
item {
name: "/m/0hnyx"
id: 558
display_name: "Pastry"
}
item {
name: "/m/0hqkz"
id: 559
display_name: "Grapefruit"
}
item {
name: "/m/0j496"
id: 560
display_name: "Band-aid"
}
item {
name: "/m/0jbk"
id: 561
display_name: "Animal"
}
item {
name: "/m/0jg57"
id: 562
display_name: "Bell pepper"
}
item {
name: "/m/0jly1"
id: 563
display_name: "Turkey"
}
item {
name: "/m/0jqgx"
id: 564
display_name: "Lily"
}
item {
name: "/m/0jwn_"
id: 565
display_name: "Pomegranate"
}
item {
name: "/m/0jy4k"
id: 566
display_name: "Doughnut"
}
item {
name: "/m/0jyfg"
id: 567
display_name: "Glasses"
}
item {
name: "/m/0k0pj"
id: 568
display_name: "Human nose"
}
item {
name: "/m/0k1tl"
id: 569
display_name: "Pen"
}
item {
name: "/m/0_k2"
id: 570
display_name: "Ant"
}
item {
name: "/m/0k4j"
id: 571
display_name: "Car"
}
item {
name: "/m/0k5j"
id: 572
display_name: "Aircraft"
}
item {
name: "/m/0k65p"
id: 573
display_name: "Human hand"
}
item {
name: "/m/0km7z"
id: 574
display_name: "Skunk"
}
item {
name: "/m/0kmg4"
id: 575
display_name: "Teddy bear"
}
item {
name: "/m/0kpqd"
id: 576
display_name: "Watermelon"
}
item {
name: "/m/0kpt_"
id: 577
display_name: "Cantaloupe"
}
item {
name: "/m/0ky7b"
id: 578
display_name: "Dishwasher"
}
item {
name: "/m/0l14j_"
id: 579
display_name: "Flute"
}
item {
name: "/m/0l3ms"
id: 580
display_name: "Balance beam"
}
item {
name: "/m/0l515"
id: 581
display_name: "Sandwich"
}
item {
name: "/m/0ll1f78"
id: 582
display_name: "Shrimp"
}
item {
name: "/m/0llzx"
id: 583
display_name: "Sewing machine"
}
item {
name: "/m/0lt4_"
id: 584
display_name: "Binoculars"
}
item {
name: "/m/0m53l"
id: 585
display_name: "Rays and skates"
}
item {
name: "/m/0mcx2"
id: 586
display_name: "Ipod"
}
item {
name: "/m/0mkg"
id: 587
display_name: "Accordion"
}
item {
name: "/m/0mw_6"
id: 588
display_name: "Willow"
}
item {
name: "/m/0n28_"
id: 589
display_name: "Crab"
}
item {
name: "/m/0nl46"
id: 590
display_name: "Crown"
}
item {
name: "/m/0nybt"
id: 591
display_name: "Seahorse"
}
item {
name: "/m/0p833"
id: 592
display_name: "Perfume"
}
item {
name: "/m/0pcr"
id: 593
display_name: "Alpaca"
}
item {
name: "/m/0pg52"
id: 594
display_name: "Taxi"
}
item {
name: "/m/0ph39"
id: 595
display_name: "Canoe"
}
item {
name: "/m/0qjjc"
id: 596
display_name: "Remote control"
}
item {
name: "/m/0qmmr"
id: 597
display_name: "Wheelchair"
}
item {
name: "/m/0wdt60w"
id: 598
display_name: "Rugby ball"
}
item {
name: "/m/0xfy"
id: 599
display_name: "Armadillo"
}
item {
name: "/m/0xzly"
id: 600
display_name: "Maracas"
}
item {
name: "/m/0zvk5"
id: 601
display_name: "Helmet"
}
research/object_detection/data_decoders/tf_example_decoder.py
View file @ 05584085
......@@ -131,7 +131,8 @@ class TfExampleDecoder(data_decoder.DataDecoder):
use_display_name=False,
dct_method='',
num_keypoints=0,
num_additional_channels=0):
num_additional_channels=0,
load_multiclass_scores=False):
"""Constructor sets keys_to_features and items_to_handlers.
Args:
......@@ -153,6 +154,8 @@ class TfExampleDecoder(data_decoder.DataDecoder):
example, the jpeg library does not have that specific option.
num_keypoints: the number of keypoints per object.
num_additional_channels: how many additional channels to use.
load_multiclass_scores: Whether to load multiclass scores associated with
boxes.
Raises:
ValueError: If `instance_mask_type` option is not one of
......@@ -205,6 +208,7 @@ class TfExampleDecoder(data_decoder.DataDecoder):
tf.VarLenFeature(tf.int64),
'image/object/weight':
tf.VarLenFeature(tf.float32),
}
# We are checking `dct_method` instead of passing it directly in order to
# ensure TF version 1.6 compatibility.
......@@ -251,7 +255,13 @@ class TfExampleDecoder(data_decoder.DataDecoder):
slim_example_decoder.Tensor('image/object/group_of')),
fields.InputDataFields.groundtruth_weights: (
slim_example_decoder.Tensor('image/object/weight')),
}
if load_multiclass_scores:
self.keys_to_features[
'image/object/class/multiclass_scores'] = tf.VarLenFeature(tf.float32)
self.items_to_handlers[fields.InputDataFields.multiclass_scores] = (
slim_example_decoder.Tensor('image/object/class/multiclass_scores'))
if num_additional_channels > 0:
self.keys_to_features[
'image/additional_channels/encoded'] = tf.FixedLenFeature(
......@@ -355,6 +365,9 @@ class TfExampleDecoder(data_decoder.DataDecoder):
shape [None, None, None] containing instance masks.
fields.InputDataFields.groundtruth_image_classes - 1D uint64 of shape
[None] containing classes for the boxes.
fields.InputDataFields.multiclass_scores - 1D float32 tensor of shape
[None * num_classes] containing flattened multiclass scores for
groundtruth boxes.
"""
serialized_example = tf.reshape(tf_example_string_tensor, shape=[])
decoder = slim_example_decoder.TFExampleDecoder(self.keys_to_features,
......
research/object_detection/data_decoders/tf_example_decoder_test.py
View file @ 05584085
......@@ -374,6 +374,43 @@ class TfExampleDecoderTest(tf.test.TestCase):
self.assertAllEqual(bbox_classes,
tensor_dict[fields.InputDataFields.groundtruth_classes])
def testDecodeMultiClassScores(self):
image_tensor = np.random.randint(256, size=(4, 5, 3)).astype(np.uint8)
encoded_jpeg = self._EncodeImage(image_tensor)
bbox_ymins = [0.0, 4.0]
bbox_xmins = [1.0, 5.0]
bbox_ymaxs = [2.0, 6.0]
bbox_xmaxs = [3.0, 7.0]
flattened_multiclass_scores = [100., 50.] + [20., 30.]
example = tf.train.Example(
features=tf.train.Features(
feature={
'image/encoded':
dataset_util.bytes_feature(encoded_jpeg),
'image/format':
dataset_util.bytes_feature('jpeg'),
'image/object/class/multiclass_scores':
dataset_util.float_list_feature(flattened_multiclass_scores
),
'image/object/bbox/ymin':
dataset_util.float_list_feature(bbox_ymins),
'image/object/bbox/xmin':
dataset_util.float_list_feature(bbox_xmins),
'image/object/bbox/ymax':
dataset_util.float_list_feature(bbox_ymaxs),
'image/object/bbox/xmax':
dataset_util.float_list_feature(bbox_xmaxs),
})).SerializeToString()
example_decoder = tf_example_decoder.TfExampleDecoder(
load_multiclass_scores=True)
tensor_dict = example_decoder.decode(tf.convert_to_tensor(example))
with self.test_session() as sess:
tensor_dict = sess.run(tensor_dict)
self.assertAllEqual(flattened_multiclass_scores,
tensor_dict[fields.InputDataFields.multiclass_scores])
def testDecodeObjectLabelNoText(self):
image_tensor = np.random.randint(256, size=(4, 5, 3)).astype(np.uint8)
encoded_jpeg = self._EncodeImage(image_tensor)
......@@ -417,6 +454,51 @@ class TfExampleDecoderTest(tf.test.TestCase):
self.assertAllEqual(bbox_classes,
tensor_dict[fields.InputDataFields.groundtruth_classes])
def testDecodeObjectLabelWithText(self):
image_tensor = np.random.randint(256, size=(4, 5, 3)).astype(np.uint8)
encoded_jpeg = self._EncodeImage(image_tensor)
bbox_classes_text = ['cat', 'dog']
# Annotation label gets overridden by labelmap id.
annotated_bbox_classes = [3, 4]
expected_bbox_classes = [1, 2]
example = tf.train.Example(
features=tf.train.Features(
feature={
'image/encoded':
dataset_util.bytes_feature(encoded_jpeg),
'image/format':
dataset_util.bytes_feature('jpeg'),
'image/object/class/text':
dataset_util.bytes_list_feature(bbox_classes_text),
'image/object/class/label':
dataset_util.int64_list_feature(annotated_bbox_classes),
})).SerializeToString()
label_map_string = """
item {
id:1
name:'cat'
}
item {
id:2
name:'dog'
}
"""
label_map_path = os.path.join(self.get_temp_dir(), 'label_map.pbtxt')
with tf.gfile.Open(label_map_path, 'wb') as f:
f.write(label_map_string)
example_decoder = tf_example_decoder.TfExampleDecoder(
label_map_proto_file=label_map_path)
tensor_dict = example_decoder.decode(tf.convert_to_tensor(example))
init = tf.tables_initializer()
with self.test_session() as sess:
sess.run(init)
tensor_dict = sess.run(tensor_dict)
self.assertAllEqual(expected_bbox_classes,
tensor_dict[fields.InputDataFields.groundtruth_classes])
def testDecodeObjectLabelUnrecognizedName(self):
image_tensor = np.random.randint(256, size=(4, 5, 3)).astype(np.uint8)
encoded_jpeg = self._EncodeImage(image_tensor)
......@@ -501,6 +583,50 @@ class TfExampleDecoderTest(tf.test.TestCase):
self.assertAllEqual([3, 1],
tensor_dict[fields.InputDataFields.groundtruth_classes])
def testDecodeObjectLabelUnrecognizedNameWithMappingWithDisplayName(self):
image_tensor = np.random.randint(256, size=(4, 5, 3)).astype(np.uint8)
encoded_jpeg = self._EncodeImage(image_tensor)
bbox_classes_text = ['cat', 'cheetah']
bbox_classes_id = [5, 6]
example = tf.train.Example(
features=tf.train.Features(
feature={
'image/encoded':
dataset_util.bytes_feature(encoded_jpeg),
'image/format':
dataset_util.bytes_feature('jpeg'),
'image/object/class/text':
dataset_util.bytes_list_feature(bbox_classes_text),
'image/object/class/label':
dataset_util.int64_list_feature(bbox_classes_id),
})).SerializeToString()
label_map_string = """
item {
name:'/m/cat'
id:3
display_name:'cat'
}
item {
name:'/m/dog'
id:1
display_name:'dog'
}
"""
label_map_path = os.path.join(self.get_temp_dir(), 'label_map.pbtxt')
with tf.gfile.Open(label_map_path, 'wb') as f:
f.write(label_map_string)
example_decoder = tf_example_decoder.TfExampleDecoder(
label_map_proto_file=label_map_path)
tensor_dict = example_decoder.decode(tf.convert_to_tensor(example))
with self.test_session() as sess:
sess.run(tf.tables_initializer())
tensor_dict = sess.run(tensor_dict)
self.assertAllEqual([3, -1],
tensor_dict[fields.InputDataFields.groundtruth_classes])
def testDecodeObjectLabelWithMappingWithName(self):
image_tensor = np.random.randint(256, size=(4, 5, 3)).astype(np.uint8)
encoded_jpeg = self._EncodeImage(image_tensor)
......
research/object_detection/eval_util.py
View file @ 05584085
......@@ -15,6 +15,7 @@
"""Common utility functions for evaluation."""
import collections
import os
import re
import time
import numpy as np
......@@ -233,7 +234,8 @@ def _run_checkpoint_once(tensor_dict,
save_graph=False,
save_graph_dir='',
losses_dict=None,
eval_export_path=None):
eval_export_path=None,
process_metrics_fn=None):
"""Evaluates metrics defined in evaluators and returns summaries.
This function loads the latest checkpoint in checkpoint_dirs and evaluates
......@@ -275,6 +277,12 @@ def _run_checkpoint_once(tensor_dict,
losses_dict: optional dictionary of scalar detection losses.
eval_export_path: Path for saving a json file that contains the detection
results in json format.
process_metrics_fn: a callback called with evaluation results after each
evaluation is done. It could be used e.g. to back up checkpoints with
best evaluation scores, or to call an external system to update evaluation
results in order to drive best hyper-parameter search. Parameters are:
int checkpoint_number, Dict[str, ObjectDetectionEvalMetrics] metrics,
str checkpoint_file path.
Returns:
global_step: the count of global steps.
......@@ -291,6 +299,7 @@ def _run_checkpoint_once(tensor_dict,
sess.run(tf.global_variables_initializer())
sess.run(tf.local_variables_initializer())
sess.run(tf.tables_initializer())
checkpoint_file = None
if restore_fn:
restore_fn(sess)
else:
......@@ -370,6 +379,15 @@ def _run_checkpoint_once(tensor_dict,
for key, value in iter(aggregate_result_losses_dict.items()):
all_evaluator_metrics['Losses/' + key] = np.mean(value)
if process_metrics_fn and checkpoint_file:
m = re.search(r'model.ckpt-(\d+)$', checkpoint_file)
if not m:
tf.logging.error('Failed to parse checkpoint number from: %s',
checkpoint_file)
else:
checkpoint_number = int(m.group(1))
process_metrics_fn(checkpoint_number, all_evaluator_metrics,
checkpoint_file)
sess.close()
return (global_step, all_evaluator_metrics)
......@@ -385,11 +403,13 @@ def repeated_checkpoint_run(tensor_dict,
num_batches=1,
eval_interval_secs=120,
max_number_of_evaluations=None,
max_evaluation_global_step=None,
master='',
save_graph=False,
save_graph_dir='',
losses_dict=None,
eval_export_path=None):
eval_export_path=None,
process_metrics_fn=None):
"""Periodically evaluates desired tensors using checkpoint_dirs or restore_fn.
This function repeatedly loads a checkpoint and evaluates a desired
......@@ -425,6 +445,7 @@ def repeated_checkpoint_run(tensor_dict,
eval_interval_secs: the number of seconds between each evaluation run.
max_number_of_evaluations: the max number of iterations of the evaluation.
If the value is left as None the evaluation continues indefinitely.
max_evaluation_global_step: global step when evaluation stops.
master: the location of the Tensorflow session.
save_graph: whether or not the Tensorflow graph is saved as a pbtxt file.
save_graph_dir: where to save on disk the Tensorflow graph. If store_graph
......@@ -432,6 +453,12 @@ def repeated_checkpoint_run(tensor_dict,
losses_dict: optional dictionary of scalar detection losses.
eval_export_path: Path for saving a json file that contains the detection
results in json format.
process_metrics_fn: a callback called with evaluation results after each
evaluation is done. It could be used e.g. to back up checkpoints with
best evaluation scores, or to call an external system to update evaluation
results in order to drive best hyper-parameter search. Parameters are:
int checkpoint_number, Dict[str, ObjectDetectionEvalMetrics] metrics,
str checkpoint_file path.
Returns:
metrics: A dictionary containing metric names and values in the latest
......@@ -443,7 +470,10 @@ def repeated_checkpoint_run(tensor_dict,
"""
if max_number_of_evaluations and max_number_of_evaluations <= 0:
raise ValueError(
'`number_of_steps` must be either None or a positive number.')
'`max_number_of_evaluations` must be either None or a positive number.')
if max_evaluation_global_step and max_evaluation_global_step <= 0:
raise ValueError(
'`max_evaluation_global_step` must be either None or positive.')
if not checkpoint_dirs:
raise ValueError('`checkpoint_dirs` must have at least one entry.')
......@@ -475,8 +505,13 @@ def repeated_checkpoint_run(tensor_dict,
save_graph,
save_graph_dir,
losses_dict=losses_dict,
eval_export_path=eval_export_path)
eval_export_path=eval_export_path,
process_metrics_fn=process_metrics_fn)
write_metrics(metrics, global_step, summary_dir)
if (max_evaluation_global_step and
global_step >= max_evaluation_global_step):
tf.logging.info('Finished evaluation!')
break
number_of_evaluations += 1
if (max_number_of_evaluations and
......
research/object_detection/export_inference_graph.py
View file @ 05584085
......@@ -39,6 +39,12 @@ and the following output nodes returned by the model.postprocess(..):
[batch, num_boxes] containing class scores for the detections.
* `detection_classes`: Outputs float32 tensors of the form
[batch, num_boxes] containing classes for the detections.
* `raw_detection_boxes`: Outputs float32 tensors of the form
[batch, raw_num_boxes, 4] containing detection boxes without
post-processing.
* `raw_detection_scores`: Outputs float32 tensors of the form
[batch, raw_num_boxes, num_classes_with_background] containing class score
logits for raw detection boxes.
* `detection_masks`: Outputs float32 tensors of the form
[batch, num_boxes, mask_height, mask_width] containing predicted instance
masks for each box if its present in the dictionary of postprocessed
......
research/object_detection/export_tflite_ssd_graph_lib.py
View file @ 05584085
......@@ -154,7 +154,9 @@ def export_tflite_graph(pipeline_config,
max_detections,
max_classes_per_detection,
detections_per_class=100,
use_regular_nms=False):
use_regular_nms=False,
binary_graph_name='tflite_graph.pb',
txt_graph_name='tflite_graph.pbtxt'):
"""Exports a tflite compatible graph and anchors for ssd detection model.
Anchors are written to a tensor and tflite compatible graph
......@@ -174,6 +176,8 @@ def export_tflite_graph(pipeline_config,
for NonMaxSuppression per class
use_regular_nms: Flag to set postprocessing op to use Regular NMS instead
of Fast NMS.
binary_graph_name: Name of the exported graph file in binary format.
txt_graph_name: Name of the exported graph file in text format.
Raises:
ValueError: if the pipeline config contains models other than ssd or uses an
......@@ -304,9 +308,9 @@ def export_tflite_graph(pipeline_config,
# Return frozen without adding post-processing custom op
transformed_graph_def = frozen_graph_def
binary_graph = os.path.join(output_dir, 'tflite_graph.pb')
binary_graph = os.path.join(output_dir, binary_graph_name)
with tf.gfile.GFile(binary_graph, 'wb') as f:
f.write(transformed_graph_def.SerializeToString())
txt_graph = os.path.join(output_dir, 'tflite_graph.pbtxt')
txt_graph = os.path.join(output_dir, txt_graph_name)
with tf.gfile.GFile(txt_graph, 'w') as f:
f.write(str(transformed_graph_def))
research/object_detection/exporter.py
View file @ 05584085
......@@ -19,11 +19,7 @@ import tempfile
import tensorflow as tf
from tensorflow.contrib.quantize.python import graph_matcher
from tensorflow.core.protobuf import saver_pb2
from tensorflow.python.client import session
from tensorflow.python.platform import gfile
from tensorflow.python.saved_model import signature_constants
from tensorflow.python.tools import freeze_graph
from tensorflow.python.training import saver as saver_lib
from tensorflow.python.tools import freeze_graph # pylint: disable=g-direct-tensorflow-import
from object_detection.builders import graph_rewriter_builder
from object_detection.builders import model_builder
from object_detection.core import standard_fields as fields
......@@ -73,7 +69,8 @@ def rewrite_nn_resize_op(is_quantized=False):
nn_resize = tf.image.resize_nearest_neighbor(
projection_op.outputs[0],
add_op.outputs[0].shape.dims[1:3],
align_corners=False)
align_corners=False,
name=os.path.split(reshape_2_op.name)[0] + '/resize_nearest_neighbor')
for index, op_input in enumerate(add_op.inputs):
if op_input == reshape_2_op.outputs[0]:
......@@ -207,6 +204,8 @@ def add_output_tensor_nodes(postprocessed_tensors,
label_id_offset = 1
boxes = postprocessed_tensors.get(detection_fields.detection_boxes)
scores = postprocessed_tensors.get(detection_fields.detection_scores)
raw_boxes = postprocessed_tensors.get(detection_fields.raw_detection_boxes)
raw_scores = postprocessed_tensors.get(detection_fields.raw_detection_scores)
classes = postprocessed_tensors.get(
detection_fields.detection_classes) + label_id_offset
keypoints = postprocessed_tensors.get(detection_fields.detection_keypoints)
......@@ -221,6 +220,12 @@ def add_output_tensor_nodes(postprocessed_tensors,
classes, name=detection_fields.detection_classes)
outputs[detection_fields.num_detections] = tf.identity(
num_detections, name=detection_fields.num_detections)
if raw_boxes is not None:
outputs[detection_fields.raw_detection_boxes] = tf.identity(
raw_boxes, name=detection_fields.raw_detection_boxes)
if raw_scores is not None:
outputs[detection_fields.raw_detection_scores] = tf.identity(
raw_scores, name=detection_fields.raw_detection_scores)
if keypoints is not None:
outputs[detection_fields.detection_keypoints] = tf.identity(
keypoints, name=detection_fields.detection_keypoints)
......@@ -252,7 +257,7 @@ def write_saved_model(saved_model_path,
outputs: A tensor dictionary containing the outputs of a DetectionModel.
"""
with tf.Graph().as_default():
with session.Session() as sess:
with tf.Session() as sess:
tf.import_graph_def(frozen_graph_def, name='')
......@@ -268,12 +273,15 @@ def write_saved_model(saved_model_path,
tf.saved_model.signature_def_utils.build_signature_def(
inputs=tensor_info_inputs,
outputs=tensor_info_outputs,
method_name=signature_constants.PREDICT_METHOD_NAME))
method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
))
builder.add_meta_graph_and_variables(
sess, [tf.saved_model.tag_constants.SERVING],
sess,
[tf.saved_model.tag_constants.SERVING],
signature_def_map={
signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
tf.saved_model.signature_constants
.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
detection_signature,
},
)
......@@ -289,9 +297,9 @@ def write_graph_and_checkpoint(inference_graph_def,
node.device = ''
with tf.Graph().as_default():
tf.import_graph_def(inference_graph_def, name='')
with session.Session() as sess:
saver = saver_lib.Saver(saver_def=input_saver_def,
save_relative_paths=True)
with tf.Session() as sess:
saver = tf.train.Saver(
saver_def=input_saver_def, save_relative_paths=True)
saver.restore(sess, trained_checkpoint_prefix)
saver.save(sess, model_path)
......@@ -308,8 +316,8 @@ def _get_outputs_from_inputs(input_tensors, detection_model,
output_collection_name)
def _build_detection_graph(input_type, detection_model, input_shape,
output_collection_name, graph_hook_fn):
def build_detection_graph(input_type, detection_model, input_shape,
output_collection_name, graph_hook_fn):
"""Build the detection graph."""
if input_type not in input_placeholder_fn_map:
raise ValueError('Unknown input type: {}'.format(input_type))
......@@ -343,7 +351,8 @@ def _export_inference_graph(input_type,
input_shape=None,
output_collection_name='inference_op',
graph_hook_fn=None,
write_inference_graph=False):
write_inference_graph=False,
temp_checkpoint_prefix=''):
"""Export helper."""
tf.gfile.MakeDirs(output_directory)
frozen_graph_path = os.path.join(output_directory,
......@@ -351,7 +360,7 @@ def _export_inference_graph(input_type,
saved_model_path = os.path.join(output_directory, 'saved_model')
model_path = os.path.join(output_directory, 'model.ckpt')
outputs, placeholder_tensor = _build_detection_graph(
outputs, placeholder_tensor = build_detection_graph(
input_type=input_type,
detection_model=detection_model,
input_shape=input_shape,
......@@ -361,12 +370,13 @@ def _export_inference_graph(input_type,
profile_inference_graph(tf.get_default_graph())
saver_kwargs = {}
if use_moving_averages:
# This check is to be compatible with both version of SaverDef.
if os.path.isfile(trained_checkpoint_prefix):
saver_kwargs['write_version'] = saver_pb2.SaverDef.V1
temp_checkpoint_prefix = tempfile.NamedTemporaryFile().name
else:
temp_checkpoint_prefix = tempfile.mkdtemp()
if not temp_checkpoint_prefix:
# This check is to be compatible with both version of SaverDef.
if os.path.isfile(trained_checkpoint_prefix):
saver_kwargs['write_version'] = saver_pb2.SaverDef.V1
temp_checkpoint_prefix = tempfile.NamedTemporaryFile().name
else:
temp_checkpoint_prefix = tempfile.mkdtemp()
replace_variable_values_with_moving_averages(
tf.get_default_graph(), trained_checkpoint_prefix,
temp_checkpoint_prefix)
......@@ -388,7 +398,7 @@ def _export_inference_graph(input_type,
'inference_graph.pbtxt')
for node in inference_graph_def.node:
node.device = ''
with gfile.GFile(inference_graph_path, 'wb') as f:
with tf.gfile.GFile(inference_graph_path, 'wb') as f:
f.write(str(inference_graph_def))
if additional_output_tensor_names is not None:
......@@ -486,4 +496,3 @@ def profile_inference_graph(graph):
tf.contrib.tfprof.model_analyzer.print_model_analysis(
graph,
tfprof_options=tfprof_flops_option)
research/object_detection/exporter_test.py
View file @ 05584085
......@@ -61,7 +61,14 @@ class FakeModel(model.DetectionModel):
[0.9, 0.0]], tf.float32),
'detection_classes': tf.constant([[0, 1],
[1, 0]], tf.float32),
'num_detections': tf.constant([2, 1], tf.float32)
'num_detections': tf.constant([2, 1], tf.float32),
'raw_detection_boxes': tf.constant([[[0.0, 0.0, 0.5, 0.5],
[0.5, 0.5, 0.8, 0.8]],
[[0.5, 0.5, 1.0, 1.0],
[0.0, 0.5, 0.0, 0.5]]],
tf.float32),
'raw_detection_scores': tf.constant([[0.7, 0.6],
[0.9, 0.5]], tf.float32),
}
if self._add_detection_keypoints:
postprocessed_tensors['detection_keypoints'] = tf.constant(
......@@ -612,7 +619,7 @@ class ExportInferenceGraphTest(tf.test.TestCase):
pipeline_config.eval_config.use_moving_averages = False
detection_model = model_builder.build(pipeline_config.model,
is_training=False)
outputs, _ = exporter._build_detection_graph(
outputs, _ = exporter.build_detection_graph(
input_type='tf_example',
detection_model=detection_model,
input_shape=None,
......@@ -760,7 +767,7 @@ class ExportInferenceGraphTest(tf.test.TestCase):
pipeline_config.eval_config.use_moving_averages = False
detection_model = model_builder.build(pipeline_config.model,
is_training=False)
outputs, placeholder_tensor = exporter._build_detection_graph(
outputs, placeholder_tensor = exporter.build_detection_graph(
input_type='tf_example',
detection_model=detection_model,
input_shape=None,
......@@ -893,7 +900,7 @@ class ExportInferenceGraphTest(tf.test.TestCase):
pipeline_config.eval_config.use_moving_averages = False
detection_model = model_builder.build(pipeline_config.model,
is_training=False)
exporter._build_detection_graph(
exporter.build_detection_graph(
input_type='tf_example',
detection_model=detection_model,
input_shape=None,
......@@ -917,13 +924,16 @@ class ExportInferenceGraphTest(tf.test.TestCase):
tf_example = od_graph.get_tensor_by_name('tf_example:0')
boxes = od_graph.get_tensor_by_name('detection_boxes:0')
scores = od_graph.get_tensor_by_name('detection_scores:0')
raw_boxes = od_graph.get_tensor_by_name('raw_detection_boxes:0')
raw_scores = od_graph.get_tensor_by_name('raw_detection_scores:0')
classes = od_graph.get_tensor_by_name('detection_classes:0')
keypoints = od_graph.get_tensor_by_name('detection_keypoints:0')
masks = od_graph.get_tensor_by_name('detection_masks:0')
num_detections = od_graph.get_tensor_by_name('num_detections:0')
(boxes_np, scores_np, classes_np, keypoints_np, masks_np,
num_detections_np) = sess.run(
[boxes, scores, classes, keypoints, masks, num_detections],
(boxes_np, scores_np, raw_boxes_np, raw_scores_np, classes_np,
keypoints_np, masks_np, num_detections_np) = sess.run(
[boxes, scores, raw_boxes, raw_scores, classes, keypoints, masks,
num_detections],
feed_dict={tf_example: tf_example_np})
self.assertAllClose(boxes_np, [[[0.0, 0.0, 0.5, 0.5],
[0.5, 0.5, 0.8, 0.8]],
......@@ -931,6 +941,12 @@ class ExportInferenceGraphTest(tf.test.TestCase):
[0.0, 0.0, 0.0, 0.0]]])
self.assertAllClose(scores_np, [[0.7, 0.6],
[0.9, 0.0]])
self.assertAllClose(raw_boxes_np, [[[0.0, 0.0, 0.5, 0.5],
[0.5, 0.5, 0.8, 0.8]],
[[0.5, 0.5, 1.0, 1.0],
[0.0, 0.5, 0.0, 0.5]]])
self.assertAllClose(raw_scores_np, [[0.7, 0.6],
[0.9, 0.5]])
self.assertAllClose(classes_np, [[1, 2],
[2, 1]])
self.assertAllClose(keypoints_np, np.arange(48).reshape([2, 2, 6, 2]))
......
research/object_detection/g3doc/detection_model_zoo.md
View file @ 05584085
......@@ -57,7 +57,7 @@ Some remarks on frozen inference graphs:
a detector (and discarding the part past that point), which negatively impacts
standard mAP metrics.
* Our frozen inference graphs are generated using the
[v1.8.0](https://github.com/tensorflow/tensorflow/tree/v1.8.0)
[v1.12.0](https://github.com/tensorflow/tensorflow/tree/v1.12.0)
release version of Tensorflow and we do not guarantee that these will work
with other versions; this being said, each frozen inference graph can be
regenerated using your current version of Tensorflow by re-running the
......@@ -78,7 +78,7 @@ Some remarks on frozen inference graphs:
| [ssd_mobilenet_v1_fpn_coco ☆](http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz) | 56 | 32 | Boxes |
| [ssd_resnet_50_fpn_coco ☆](http://download.tensorflow.org/models/object_detection/ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz) | 76 | 35 | Boxes |
| [ssd_mobilenet_v2_coco](http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz) | 31 | 22 | Boxes |
| [ssd_mobilenet_v2_quantized_coco](http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_quantized_300x300_coco_2018_09_14.tar.gz) | 29 | 22 | Boxes |
| [ssd_mobilenet_v2_quantized_coco](http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03.tar.gz) | 29 | 22 | Boxes |
| [ssdlite_mobilenet_v2_coco](http://download.tensorflow.org/models/object_detection/ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz) | 27 | 22 | Boxes |
| [ssd_inception_v2_coco](http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_2018_01_28.tar.gz) | 42 | 24 | Boxes |
| [faster_rcnn_inception_v2_coco](http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_v2_coco_2018_01_28.tar.gz) | 58 | 28 | Boxes |
......@@ -110,10 +110,15 @@ Model name
Model name | Speed (ms) | Open Images mAP@0.5[^2] | Outputs
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------: | :---------------------: | :-----:
[faster_rcnn_inception_resnet_v2_atrous_oidv2](http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_resnet_v2_atrous_oid_2018_01_28.tar.gz) | 727 | 37 | Boxes
[faster_rcnn_inception_resnet_v2_atrous_oidv2](http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_resnet_v2_atrous_oid_2018_01_28.tar.gz) | 727 | 37 | Boxes
[faster_rcnn_inception_resnet_v2_atrous_lowproposals_oidv2](http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_resnet_v2_atrous_lowproposals_oid_2018_01_28.tar.gz) | 347 | | Boxes
[facessd_mobilenet_v2_quantized_open_image_v4](http://download.tensorflow.org/models/object_detection/facessd_mobilenet_v2_quantized_320x320_open_image_v4.tar.gz) [^3] | 20 | 73 (faces) | Boxes
Model name | Speed (ms) | Open Images mAP@0.5[^4] | Outputs
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------: | :---------------------: | :-----:
[faster_rcnn_inception_resnet_v2_atrous_oidv4](http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_resnet_v2_atrous_oid_v4_2018_12_12.tar.gz) | 425 | 54 | Boxes
[ssd_mobilenetv2_oidv4](http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_oid_v4_2018_12_12.tar.gz) | 89 | 36 | Boxes
[ssd_resnet_101_fpn_oidv4](http://download.tensorflow.org/models/object_detection/ssd_resnet101_v1_fpn_shared_box_predictor_oid_512x512_sync_2019_01_20.tar.gz) | 237 | 38 | Boxes
## iNaturalist Species-trained models
Model name | Speed (ms) | Pascal mAP@0.5 | Outputs
......@@ -129,8 +134,10 @@ Model name
[faster_rcnn_resnet101_ava_v2.1](http://download.tensorflow.org/models/object_detection/faster_rcnn_resnet101_ava_v2.1_2018_04_30.tar.gz) | 93 | 11 | Boxes
[^1]: See [MSCOCO evaluation protocol](http://cocodataset.org/#detections-eval).
[^1]: See [MSCOCO evaluation protocol](http://cocodataset.org/#detections-eval). The COCO mAP numbers here are evaluated on COCO 14 minival set (note that our split is different from COCO 17 Val). A full list of image ids used in our split could be fould [here](https://github.com/tensorflow/models/blob/master/research/object_detection/data/mscoco_minival_ids.txt).
[^2]: This is PASCAL mAP with a slightly different way of true positives computation: see [Open Images evaluation protocol](evaluation_protocols.md#open-images).
[^2]: This is PASCAL mAP with a slightly different way of true positives computation: see [Open Images evaluation protocols](evaluation_protocols.md), oid_V2_detection_metrics.
[^3]: Non-face boxes are dropped during training and non-face groundtruth boxes are ignored when evaluating.
[^4]: This is Open Images Challenge metric: see [Open Images evaluation protocols](evaluation_protocols.md), oid_challenge_detection_metrics.
research/object_detection/g3doc/installation.md
View file @ 05584085
......@@ -11,7 +11,7 @@ Tensorflow Object Detection API depends on the following libraries:
* tf Slim (which is included in the "tensorflow/models/research/" checkout)
* Jupyter notebook
* Matplotlib
* Tensorflow (>=1.9.0)
* Tensorflow (>=1.12.0)
* Cython
* contextlib2
* cocoapi
......
research/object_detection/g3doc/running_on_cloud.md
View file @ 05584085
......@@ -44,7 +44,7 @@ job using GPUs. A sample YAML file is given below:
```
trainingInput:
runtimeVersion: "1.9"
runtimeVersion: "1.12"
scaleTier: CUSTOM
masterType: standard_gpu
workerCount: 9
......@@ -73,7 +73,7 @@ following command:
```bash
# From tensorflow/models/research/
gcloud ml-engine jobs submit training object_detection_`date +%m_%d_%Y_%H_%M_%S` \
--runtime-version 1.9 \
--runtime-version 1.12 \
--job-dir=gs://${MODEL_DIR} \
--packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,/tmp/pycocotools/pycocotools-2.0.tar.gz \
--module-name object_detection.model_main \
......@@ -93,7 +93,7 @@ Google Cloud Storage.
Users can monitor the progress of their training job on the [ML Engine
Dashboard](https://console.cloud.google.com/mlengine/jobs).
Note: This sample is supported for use with 1.9 runtime version.
Note: This sample is supported for use with 1.12 runtime version.
## Running a TPU Training Job on CMLE
......@@ -105,7 +105,7 @@ gcloud ml-engine jobs submit training `whoami`_object_detection_`date +%m_%d_%Y_
--job-dir=gs://${MODEL_DIR} \
--packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,/tmp/pycocotools/pycocotools-2.0.tar.gz \
--module-name object_detection.model_tpu_main \
--runtime-version 1.9 \
--runtime-version 1.12 \
--scale-tier BASIC_TPU \
--region us-central1 \
-- \
......@@ -133,7 +133,7 @@ job:
```bash
gcloud ml-engine jobs submit training object_detection_eval_`date +%m_%d_%Y_%H_%M_%S` \
--runtime-version 1.9 \
--runtime-version 1.12 \
--job-dir=gs://${MODEL_DIR} \
--packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,/tmp/pycocotools/pycocotools-2.0.tar.gz \
--module-name object_detection.model_main \
......
research/object_detection/g3doc/running_pets.md
View file @ 05584085
......@@ -208,7 +208,7 @@ For running the training Cloud ML job, we'll configure the cluster to use 5
training jobs and three parameters servers. The
configuration file can be found at `object_detection/samples/cloud/cloud.yml`.
Note: The code sample below is supported for use with 1.9 runtime version.
Note: The code sample below is supported for use with 1.12 runtime version.
To start training and evaluation, execute the following command from the
`tensorflow/models/research/` directory:
......@@ -216,7 +216,7 @@ To start training and evaluation, execute the following command from the
```bash
# From tensorflow/models/research/
gcloud ml-engine jobs submit training `whoami`_object_detection_pets_`date +%m_%d_%Y_%H_%M_%S` \
--runtime-version 1.9 \
--runtime-version 1.12 \
--job-dir=gs://${YOUR_GCS_BUCKET}/model_dir \
--packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,/tmp/pycocotools/pycocotools-2.0.tar.gz \
--module-name object_detection.model_main \
......
research/object_detection/g3doc/using_your_own_dataset.md
View file @ 05584085
......@@ -76,7 +76,7 @@ def create_cat_tf_example(encoded_cat_image_data):
'image/width': dataset_util.int64_feature(width),
'image/filename': dataset_util.bytes_feature(filename),
'image/source_id': dataset_util.bytes_feature(filename),
'image/encoded': dataset_util.bytes_feature(encoded_cat_image_data),
'image/encoded': dataset_util.bytes_feature(encoded_image_data),
'image/format': dataset_util.bytes_feature(image_format),
'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),
'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),
......
research/object_detection/inputs.py
View file @ 05584085
......@@ -53,6 +53,7 @@ def transform_input_data(tensor_dict,
data_augmentation_fn=None,
merge_multiple_boxes=False,
retain_original_image=False,
use_multiclass_scores=False,
use_bfloat16=False):
"""A single function that is responsible for all input data transformations.
......@@ -87,25 +88,37 @@ def transform_input_data(tensor_dict,
and classes for a given image if the boxes are exactly the same.
retain_original_image: (optional) whether to retain original image in the
output dictionary.
use_multiclass_scores: whether to use multiclass scores as
class targets instead of one-hot encoding of `groundtruth_classes`.
use_bfloat16: (optional) a bool, whether to use bfloat16 in training.
Returns:
A dictionary keyed by fields.InputDataFields containing the tensors obtained
after applying all the transformations.
"""
# Reshape flattened multiclass scores tensor into a 2D tensor of shape
# [num_boxes, num_classes].
if fields.InputDataFields.multiclass_scores in tensor_dict:
tensor_dict[fields.InputDataFields.multiclass_scores] = tf.reshape(
tensor_dict[fields.InputDataFields.multiclass_scores], [
tf.shape(tensor_dict[fields.InputDataFields.groundtruth_boxes])[0],
num_classes
])
if fields.InputDataFields.groundtruth_boxes in tensor_dict:
tensor_dict = util_ops.filter_groundtruth_with_nan_box_coordinates(
tensor_dict)
if fields.InputDataFields.image_additional_channels in tensor_dict:
channels = tensor_dict[fields.InputDataFields.image_additional_channels]
tensor_dict[fields.InputDataFields.image] = tf.concat(
[tensor_dict[fields.InputDataFields.image], channels], axis=2)
tensor_dict = util_ops.filter_unrecognized_classes(tensor_dict)
if retain_original_image:
tensor_dict[fields.InputDataFields.original_image] = tf.cast(
image_resizer_fn(tensor_dict[fields.InputDataFields.image], None)[0],
tf.uint8)
if fields.InputDataFields.image_additional_channels in tensor_dict:
channels = tensor_dict[fields.InputDataFields.image_additional_channels]
tensor_dict[fields.InputDataFields.image] = tf.concat(
[tensor_dict[fields.InputDataFields.image], channels], axis=2)
# Apply data augmentation ops.
if data_augmentation_fn is not None:
tensor_dict = data_augmentation_fn(tensor_dict)
......@@ -136,6 +149,11 @@ def transform_input_data(tensor_dict,
tensor_dict[fields.InputDataFields.groundtruth_classes] = tf.one_hot(
zero_indexed_groundtruth_classes, num_classes)
if use_multiclass_scores:
tensor_dict[fields.InputDataFields.groundtruth_classes] = tensor_dict[
fields.InputDataFields.multiclass_scores]
tensor_dict.pop(fields.InputDataFields.multiclass_scores, None)
if fields.InputDataFields.groundtruth_confidences in tensor_dict:
groundtruth_confidences = tensor_dict[
fields.InputDataFields.groundtruth_confidences]
......@@ -172,6 +190,9 @@ def pad_input_data_to_static_shapes(tensor_dict, max_num_boxes, num_classes,
spatial_image_shape=None):
"""Pads input tensors to static shapes.
In case num_additional_channels > 0, we assume that the additional channels
have already been concatenated to the base image.
Args:
tensor_dict: Tensor dictionary of input data
max_num_boxes: Max number of groundtruth boxes needed to compute shapes for
......@@ -186,7 +207,8 @@ def pad_input_data_to_static_shapes(tensor_dict, max_num_boxes, num_classes,
tensors in the dataset.
Raises:
ValueError: If groundtruth classes is neither rank 1 nor rank 2.
ValueError: If groundtruth classes is neither rank 1 nor rank 2, or if we
detect that additional channels have not been concatenated yet.
"""
if not spatial_image_shape or spatial_image_shape == [-1, -1]:
......@@ -198,14 +220,27 @@ def pad_input_data_to_static_shapes(tensor_dict, max_num_boxes, num_classes,
if fields.InputDataFields.image_additional_channels in tensor_dict:
num_additional_channels = tensor_dict[
fields.InputDataFields.image_additional_channels].shape[2].value
num_image_channels = 3
# We assume that if num_additional_channels > 0, then it has already been
# concatenated to the base image (but not the ground truth).
num_channels = 3
if fields.InputDataFields.image in tensor_dict:
num_image_channels = tensor_dict[fields.InputDataFields
.image].shape[2].value
num_channels = tensor_dict[fields.InputDataFields.image].shape[2].value
if num_additional_channels:
if num_additional_channels >= num_channels:
raise ValueError(
'Image must be already concatenated with additional channels.')
if (fields.InputDataFields.original_image in tensor_dict and
tensor_dict[fields.InputDataFields.original_image].shape[2].value ==
num_channels):
raise ValueError(
'Image must be already concatenated with additional channels.')
padding_shapes = {
# Additional channels are merged before batching.
fields.InputDataFields.image: [
height, width, num_image_channels + num_additional_channels
height, width, num_channels
],
fields.InputDataFields.original_image_spatial_shape: [2],
fields.InputDataFields.image_additional_channels: [
......@@ -231,16 +266,14 @@ def pad_input_data_to_static_shapes(tensor_dict, max_num_boxes, num_classes,
fields.InputDataFields.groundtruth_label_types: [max_num_boxes],
fields.InputDataFields.groundtruth_label_weights: [max_num_boxes],
fields.InputDataFields.true_image_shape: [3],
fields.InputDataFields.multiclass_scores: [
max_num_boxes, num_classes + 1 if num_classes is not None else None
],
fields.InputDataFields.groundtruth_image_classes: [num_classes],
fields.InputDataFields.groundtruth_image_confidences: [num_classes],
}
if fields.InputDataFields.original_image in tensor_dict:
padding_shapes[fields.InputDataFields.original_image] = [
height, width, num_image_channels + num_additional_channels
height, width, tensor_dict[fields.InputDataFields.
original_image].shape[2].value
]
if fields.InputDataFields.groundtruth_keypoints in tensor_dict:
tensor_shape = (
......@@ -294,11 +327,14 @@ def augment_input_data(tensor_dict, data_augmentation_options):
in tensor_dict)
include_label_confidences = (fields.InputDataFields.groundtruth_confidences
in tensor_dict)
include_multiclass_scores = (fields.InputDataFields.multiclass_scores in
tensor_dict)
tensor_dict = preprocessor.preprocess(
tensor_dict, data_augmentation_options,
func_arg_map=preprocessor.get_default_func_arg_map(
include_label_weights=include_label_weights,
include_label_confidences=include_label_confidences,
include_multiclass_scores=include_multiclass_scores,
include_instance_masks=include_instance_masks,
include_keypoints=include_keypoints))
tensor_dict[fields.InputDataFields.image] = tf.squeeze(
......@@ -472,6 +508,7 @@ def create_train_input_fn(train_config, train_input_config,
data_augmentation_fn=data_augmentation_fn,
merge_multiple_boxes=train_config.merge_multiple_label_boxes,
retain_original_image=train_config.retain_original_images,
use_multiclass_scores=train_config.use_multiclass_scores,
use_bfloat16=train_config.use_bfloat16)
tensor_dict = pad_input_data_to_static_shapes(
......
research/object_detection/inputs_test.py
View file @ 05584085
......@@ -105,6 +105,48 @@ class InputsTest(test_case.TestCase, parameterized.TestCase):
tf.float32,
labels[fields.InputDataFields.groundtruth_confidences].dtype)
def test_faster_rcnn_resnet50_train_input_with_additional_channels(self):
"""Tests the training input function for FasterRcnnResnet50."""
configs = _get_configs_for_model('faster_rcnn_resnet50_pets')
model_config = configs['model']
configs['train_input_config'].num_additional_channels = 2
configs['train_config'].retain_original_images = True
model_config.faster_rcnn.num_classes = 37
train_input_fn = inputs.create_train_input_fn(
configs['train_config'], configs['train_input_config'], model_config)
features, labels = _make_initializable_iterator(train_input_fn()).get_next()
self.assertAllEqual([1, None, None, 5],
features[fields.InputDataFields.image].shape.as_list())
self.assertAllEqual(
[1, None, None, 3],
features[fields.InputDataFields.original_image].shape.as_list())
self.assertEqual(tf.float32, features[fields.InputDataFields.image].dtype)
self.assertAllEqual([1],
features[inputs.HASH_KEY].shape.as_list())
self.assertEqual(tf.int32, features[inputs.HASH_KEY].dtype)
self.assertAllEqual(
[1, 100, 4],
labels[fields.InputDataFields.groundtruth_boxes].shape.as_list())
self.assertEqual(tf.float32,
labels[fields.InputDataFields.groundtruth_boxes].dtype)
self.assertAllEqual(
[1, 100, model_config.faster_rcnn.num_classes],
labels[fields.InputDataFields.groundtruth_classes].shape.as_list())
self.assertEqual(tf.float32,
labels[fields.InputDataFields.groundtruth_classes].dtype)
self.assertAllEqual(
[1, 100],
labels[fields.InputDataFields.groundtruth_weights].shape.as_list())
self.assertEqual(tf.float32,
labels[fields.InputDataFields.groundtruth_weights].dtype)
self.assertAllEqual(
[1, 100, model_config.faster_rcnn.num_classes],
labels[fields.InputDataFields.groundtruth_confidences].shape.as_list())
self.assertEqual(
tf.float32,
labels[fields.InputDataFields.groundtruth_confidences].dtype)
@parameterized.parameters(
{'eval_batch_size': 1},
{'eval_batch_size': 8}
......@@ -595,6 +637,72 @@ class DataTransformationFnTest(test_case.TestCase):
transformed_inputs[fields.InputDataFields.groundtruth_confidences],
[[0, 0, 1], [1, 0, 0]])
def test_returns_correct_labels_with_unrecognized_class(self):
tensor_dict = {
fields.InputDataFields.image:
tf.constant(np.random.rand(4, 4, 3).astype(np.float32)),
fields.InputDataFields.groundtruth_boxes:
tf.constant(
np.array([[0, 0, 1, 1], [.2, .2, 4, 4], [.5, .5, 1, 1]],
np.float32)),
fields.InputDataFields.groundtruth_area:
tf.constant(np.array([.5, .4, .3])),
fields.InputDataFields.groundtruth_classes:
tf.constant(np.array([3, -1, 1], np.int32)),
fields.InputDataFields.groundtruth_keypoints:
tf.constant(
np.array([[[.1, .1]], [[.2, .2]], [[.5, .5]]],
np.float32)),
fields.InputDataFields.groundtruth_keypoint_visibilities:
tf.constant([True, False, True]),
fields.InputDataFields.groundtruth_instance_masks:
tf.constant(np.random.rand(3, 4, 4).astype(np.float32)),
fields.InputDataFields.groundtruth_is_crowd:
tf.constant([False, True, False]),
fields.InputDataFields.groundtruth_difficult:
tf.constant(np.array([0, 0, 1], np.int32))
}
num_classes = 3
input_transformation_fn = functools.partial(
inputs.transform_input_data,
model_preprocess_fn=_fake_model_preprocessor_fn,
image_resizer_fn=_fake_image_resizer_fn,
num_classes=num_classes)
with self.test_session() as sess:
transformed_inputs = sess.run(
input_transformation_fn(tensor_dict=tensor_dict))
self.assertAllClose(
transformed_inputs[fields.InputDataFields.groundtruth_classes],
[[0, 0, 1], [1, 0, 0]])
self.assertAllEqual(
transformed_inputs[fields.InputDataFields.num_groundtruth_boxes], 2)
self.assertAllClose(
transformed_inputs[fields.InputDataFields.groundtruth_area], [.5, .3])
self.assertAllEqual(
transformed_inputs[fields.InputDataFields.groundtruth_confidences],
[[0, 0, 1], [1, 0, 0]])
self.assertAllClose(
transformed_inputs[fields.InputDataFields.groundtruth_boxes],
[[0, 0, 1, 1], [.5, .5, 1, 1]])
self.assertAllClose(
transformed_inputs[fields.InputDataFields.groundtruth_keypoints],
[[[.1, .1]], [[.5, .5]]])
self.assertAllEqual(
transformed_inputs[
fields.InputDataFields.groundtruth_keypoint_visibilities],
[True, True])
self.assertAllEqual(
transformed_inputs[
fields.InputDataFields.groundtruth_instance_masks].shape, [2, 4, 4])
self.assertAllEqual(
transformed_inputs[fields.InputDataFields.groundtruth_is_crowd],
[False, False])
self.assertAllEqual(
transformed_inputs[fields.InputDataFields.groundtruth_difficult],
[0, 1])
def test_returns_correct_merged_boxes(self):
tensor_dict = {
fields.InputDataFields.image:
......@@ -885,7 +993,7 @@ class PadInputDataToStaticShapesFnTest(test_case.TestCase):
def test_images_and_additional_channels(self):
input_tensor_dict = {
fields.InputDataFields.image:
tf.placeholder(tf.float32, [None, None, 3]),
tf.placeholder(tf.float32, [None, None, 5]),
fields.InputDataFields.image_additional_channels:
tf.placeholder(tf.float32, [None, None, 2]),
}
......@@ -895,6 +1003,8 @@ class PadInputDataToStaticShapesFnTest(test_case.TestCase):
num_classes=3,
spatial_image_shape=[5, 6])
# pad_input_data_to_static_shape assumes that image is already concatenated
# with additional channels.
self.assertAllEqual(
padded_tensor_dict[fields.InputDataFields.image].shape.as_list(),
[5, 6, 5])
......@@ -902,6 +1012,22 @@ class PadInputDataToStaticShapesFnTest(test_case.TestCase):
padded_tensor_dict[fields.InputDataFields.image_additional_channels]
.shape.as_list(), [5, 6, 2])
def test_images_and_additional_channels_errors(self):
input_tensor_dict = {
fields.InputDataFields.image:
tf.placeholder(tf.float32, [None, None, 3]),
fields.InputDataFields.image_additional_channels:
tf.placeholder(tf.float32, [None, None, 2]),
fields.InputDataFields.original_image:
tf.placeholder(tf.float32, [None, None, 3]),
}
with self.assertRaises(ValueError):
_ = inputs.pad_input_data_to_static_shapes(
tensor_dict=input_tensor_dict,
max_num_boxes=3,
num_classes=3,
spatial_image_shape=[5, 6])
def test_gray_images(self):
input_tensor_dict = {
fields.InputDataFields.image:
......@@ -920,10 +1046,12 @@ class PadInputDataToStaticShapesFnTest(test_case.TestCase):
def test_gray_images_and_additional_channels(self):
input_tensor_dict = {
fields.InputDataFields.image:
tf.placeholder(tf.float32, [None, None, 1]),
tf.placeholder(tf.float32, [None, None, 3]),
fields.InputDataFields.image_additional_channels:
tf.placeholder(tf.float32, [None, None, 2]),
}
# pad_input_data_to_static_shape assumes that image is already concatenated
# with additional channels.
padded_tensor_dict = inputs.pad_input_data_to_static_shapes(
tensor_dict=input_tensor_dict,
max_num_boxes=3,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment