Skip to content

GitLab

Commit b0ccdb11 authored by Shixin Luo's avatar Shixin Luo
Browse files

resolve conflict with master

parents e61588cd 1611a8c5
Hide whitespace changes
Inline Side-by-side
Showing with 973 additions and 22 deletions
official/vision/keras_cv/LICENSE 0 → 100644
View file @ b0ccdb11
Copyright 2020 The TensorFlow Authors. All rights reserved.
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2015, The TensorFlow Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
\ No newline at end of file
official/vision/keras_cv/README.md 0 → 100644
View file @ b0ccdb11
# keras-cv
## Losses
* [FocalLoss](losses/focal_loss.py) implements Focal loss as described in
["Focal Loss for Dense Object Detection"](https://arxiv.org/abs/1708.02002).
## Ops
Ops are used in data pipeline for pre-compute labels, weights.
* [IOUSimilarity](ops/iou_similarity.py) implements Intersection-Over-Union.
official/vision/keras_cv/__init__.py
View file @ b0ccdb11
......@@ -12,6 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Keras-NLP package definition."""
"""Keras-CV package definition."""
# pylint: disable=wildcard-import
from official.vision.keras_cv.losses import *
from official.vision.keras_cv import layers
from official.vision.keras_cv import losses
from official.vision.keras_cv import ops
official/vision/keras_cv/contributing.md 0 → 100644
View file @ b0ccdb11
## Contributing to KerasCV
Patches to KerasCV are welcome!
The source-of-truth repository lives under
[TF Model Garden Vision](https://github.com/tensorflow/models/official/vision/keras_cv),
and is mirrored as a read-only repository under
[keras-team/keras-cv](https://github.com/keras-team/keras-cv).
Contributions should be made as PRs to the TF Model Garden repository.
This is to ensure the codebase is rigorously tested with state-of-art models
on different accelerators.
In the long run, we will move development to the current repository `keras-team/keras-cv`.
## :heavy_check_mark: Contributor checklist
1. Ensure you have signed the [Contributor License Agreement](https://cla.developers.google.com/about/google-individual?csw=1).
* All code contributors are required to sign a Contributor License Agreement.
* Please read this [troubleshooting guide](Contributor-License-Agreements#troubleshooting-clas)
if you encounter an issue.
2. Please review the [contribution guidelines](https://github.com/tensorflow/models/wiki/How-to-contribute).
3. Check if your changes are consistent with the [TensorFlow coding style](https://www.tensorflow.org/community/contribute/code_style).
official/vision/keras_cv/layers/__init__.py 0 → 100644
View file @ b0ccdb11
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Keras-CV layers package definition."""
from official.vision.keras_cv.layers.deeplab import ASPP
official/vision/keras_cv/layers/deeplab.py 0 → 100644
View file @ b0ccdb11
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Layers for DeepLabV3."""
import tensorflow as tf
@tf.keras.utils.register_keras_serializable(package='keras_cv')
class ASPP(tf.keras.layers.Layer):
"""Implements the Atrous Spatial Pyramid Pooling.
Reference:
[Rethinking Atrous Convolution for Semantic Image Segmentation](
https://arxiv.org/pdf/1706.05587.pdf)
"""
def __init__(
self,
output_channels,
dilation_rates,
batchnorm_momentum=0.99,
dropout=0.5,
kernel_initializer='glorot_uniform',
kernel_regularizer=None,
interpolation='bilinear',
**kwargs):
"""Initializes `ASPP`.
Arguments:
output_channels: Number of channels produced by ASPP.
dilation_rates: A list of integers for parallel dilated conv.
batchnorm_momentum: A float for the momentum in BatchNorm. Defaults to
0.99.
dropout: A float for the dropout rate before output. Defaults to 0.5.
kernel_initializer: Kernel initializer for conv layers. Defaults to
`glorot_uniform`.
kernel_regularizer: Kernel regularizer for conv layers. Defaults to None.
interpolation: The interpolation method for upsampling. Defaults to
`bilinear`.
**kwargs: Other keyword arguments for the layer.
"""
super(ASPP, self).__init__(**kwargs)
self.output_channels = output_channels
self.dilation_rates = dilation_rates
self.batchnorm_momentum = batchnorm_momentum
self.dropout = dropout
self.kernel_initializer = tf.keras.initializers.get(kernel_initializer)
self.kernel_regularizer = tf.keras.regularizers.get(kernel_regularizer)
self.interpolation = interpolation
self.input_spec = tf.keras.layers.InputSpec(ndim=4)
def build(self, input_shape):
height = input_shape[1]
width = input_shape[2]
channels = input_shape[3]
self.aspp_layers = []
conv_sequential = tf.keras.Sequential([
tf.keras.layers.Conv2D(
filters=self.output_channels, kernel_size=(1, 1),
kernel_initializer=self.kernel_initializer,
kernel_regularizer=self.kernel_regularizer, use_bias=False),
tf.keras.layers.BatchNormalization(momentum=self.batchnorm_momentum),
tf.keras.layers.Activation('relu')])
self.aspp_layers.append(conv_sequential)
for dilation_rate in self.dilation_rates:
conv_sequential = tf.keras.Sequential([
tf.keras.layers.Conv2D(
filters=self.output_channels, kernel_size=(3, 3),
padding='same', kernel_regularizer=self.kernel_regularizer,
kernel_initializer=self.kernel_initializer,
dilation_rate=dilation_rate, use_bias=False),
tf.keras.layers.BatchNormalization(momentum=self.batchnorm_momentum),
tf.keras.layers.Activation('relu')])
self.aspp_layers.append(conv_sequential)
pool_sequential = tf.keras.Sequential([
tf.keras.layers.GlobalAveragePooling2D(),
tf.keras.layers.Reshape((1, 1, channels)),
tf.keras.layers.Conv2D(
filters=self.output_channels, kernel_size=(1, 1),
kernel_initializer=self.kernel_initializer,
kernel_regularizer=self.kernel_regularizer, use_bias=False),
tf.keras.layers.BatchNormalization(momentum=self.batchnorm_momentum),
tf.keras.layers.Activation('relu'),
tf.keras.layers.experimental.preprocessing.Resizing(
height, width, interpolation=self.interpolation)])
self.aspp_layers.append(pool_sequential)
self.projection = tf.keras.Sequential([
tf.keras.layers.Conv2D(
filters=self.output_channels, kernel_size=(1, 1),
kernel_initializer=self.kernel_initializer,
kernel_regularizer=self.kernel_regularizer, use_bias=False),
tf.keras.layers.BatchNormalization(momentum=self.batchnorm_momentum),
tf.keras.layers.Activation('relu'),
tf.keras.layers.Dropout(rate=self.dropout)])
def call(self, inputs, training=None):
if training is None:
training = tf.keras.backend.learning_phase()
result = []
for layer in self.aspp_layers:
result.append(layer(inputs, training=training))
result = tf.concat(result, axis=-1)
result = self.projection(result, training=training)
return result
def get_config(self):
config = {
'output_channels': self.output_channels,
'dilation_rates': self.dilation_rates,
'batchnorm_momentum': self.batchnorm_momentum,
'dropout': self.dropout,
'kernel_initializer': tf.keras.initializers.serialize(
self.kernel_initializer),
'kernel_regularizer': tf.keras.regularizers.serialize(
self.kernel_regularizer),
'interpolation': self.interpolation,
}
base_config = super(ASPP, self).get_config()
return dict(list(base_config.items()) + list(config.items()))
official/vision/beta/modeling/layers/box_sampler_test.py → official/vision/keras_cv/layers/deeplab_test.py
View file @ b0ccdb11
......@@ -12,57 +12,34 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Tests for roi_sampler.py."""
"""Tests for ASPP."""
# Import libraries
import numpy as np
import tensorflow as tf
from official.vision.beta.modeling.layers import box_sampler
from tensorflow.python.keras import keras_parameterized
from official.vision.keras_cv.layers import deeplab
class BoxSamplerTest(tf.test.TestCase):
@keras_parameterized.run_all_keras_modes
class DeeplabTest(keras_parameterized.TestCase):
def test_box_sampler(self):
positive_matches = np.array(
[[True, False, False, False, True, True, False],
[False, False, False, False, False, True, True]])
negative_matches = np.array(
[[False, True, True, True, False, False, False],
[True, True, True, True, False, False, False]])
ignored_matches = np.array(
[[False, False, False, False, False, False, True],
[False, False, False, False, True, False, False]])
def test_aspp(self):
inputs = tf.keras.Input(shape=(64, 64, 128), dtype=tf.float32)
layer = deeplab.ASPP(output_channels=256, dilation_rates=[6, 12, 18])
output = layer(inputs)
self.assertAllEqual([None, 64, 64, 256], output.shape)
sampler = box_sampler.BoxSampler(num_samples=2, foreground_fraction=0.5)
def test_aspp_invalid_shape(self):
inputs = tf.keras.Input(shape=(64, 64), dtype=tf.float32)
layer = deeplab.ASPP(output_channels=256, dilation_rates=[6, 12, 18])
with self.assertRaises(ValueError):
_ = layer(inputs)
# Runs on TPU.
strategy = tf.distribute.experimental.TPUStrategy()
with strategy.scope():
selected_indices_tpu = sampler(
positive_matches, negative_matches, ignored_matches)
self.assertEqual(2, tf.shape(selected_indices_tpu)[1])
# Runs on CPU.
selected_indices_cpu = sampler(
positive_matches, negative_matches, ignored_matches)
self.assertEqual(2, tf.shape(selected_indices_cpu)[1])
def test_serialize_deserialize(self):
kwargs = dict(
num_samples=512,
foreground_fraction=0.25,
)
sampler = box_sampler.BoxSampler(**kwargs)
expected_config = dict(kwargs)
self.assertEqual(sampler.get_config(), expected_config)
new_sampler = box_sampler.BoxSampler.from_config(
sampler.get_config())
self.assertAllEqual(sampler.get_config(), new_sampler.get_config())
def test_config_with_custom_name(self):
layer = deeplab.ASPP(256, [5], name='aspp')
config = layer.get_config()
layer_1 = deeplab.ASPP.from_config(config)
self.assertEqual(layer_1.name, layer.name)
if __name__ == '__main__':
......
official/vision/keras_cv/losses/__init__.py
View file @ b0ccdb11
......@@ -14,4 +14,4 @@
# ==============================================================================
"""Keras-CV layers package definition."""
from official.vision.keras_cv.losses.focal_loss import FocalLoss
from official.vision.keras_cv.losses.loss_utils import *
from official.vision.keras_cv.losses.loss_utils import multi_level_flatten
official/vision/keras_cv/losses/focal_loss.py
View file @ b0ccdb11
# Copyright 2018 The TensorFlow Authors. All Rights Reserved.
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
......@@ -14,11 +14,6 @@
# ==============================================================================
"""Losses used for detection models."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
# Import libraries
import tensorflow as tf
......
official/vision/keras_cv/losses/loss_utils.py
View file @ b0ccdb11
......@@ -14,7 +14,6 @@
# ==============================================================================
"""Losses utilities for detection models."""
# Import libraries
import tensorflow as tf
......
official/vision/keras_cv/ops/__init__.py 0 → 100644
View file @ b0ccdb11
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Keras-CV layers package definition."""
from official.vision.keras_cv.ops.anchor_generator import AnchorGenerator
from official.vision.keras_cv.ops.anchor_labeler import AnchorLabeler
from official.vision.keras_cv.ops.box_matcher import BoxMatcher
from official.vision.keras_cv.ops.iou_similarity import IouSimilarity
official/vision/beta/ops/experimental/anchor_generator_test.py → official/vision/keras_cv/ops/anchor_generator_test.py
View file @ b0ccdb11
......@@ -18,7 +18,7 @@ from absl.testing import parameterized
import tensorflow as tf
from tensorflow.python.distribute import combinations
from tensorflow.python.distribute import strategy_combinations
from official.vision.beta.ops.experimental import anchor_generator
from official.vision.keras_cv.ops import anchor_generator
class AnchorGeneratorTest(parameterized.TestCase, tf.test.TestCase):
......@@ -133,16 +133,16 @@ class MultiScaleAnchorGeneratorTest(parameterized.TestCase, tf.test.TestCase):
@parameterized.parameters(
# Multi scale anchor.
(5, 6, [1.0], {
5: [[[-16., -16., 48., 48.], [-16., 16., 48., 80.]],
[[16., -16., 80., 48.], [16., 16., 80., 80.]]],
6: [[[-32, -32, 96, 96]]]
'5': [[[-16., -16., 48., 48.], [-16., 16., 48., 80.]],
[[16., -16., 80., 48.], [16., 16., 80., 80.]]],
'6': [[[-32, -32, 96, 96]]]
}),)
def testAnchorGenerationDict(self, min_level, max_level, aspect_ratios,
expected_boxes):
image_size = [64, 64]
levels = range(min_level, max_level + 1)
anchor_sizes = dict((level, 2**(level + 1)) for level in levels)
strides = dict((level, 2**level) for level in levels)
anchor_sizes = dict((str(level), 2**(level + 1)) for level in levels)
strides = dict((str(level), 2**level) for level in levels)
anchor_gen = anchor_generator.AnchorGenerator(
anchor_sizes=anchor_sizes,
scales=[1.],
......
official/vision/keras_cv/ops/anchor_labeler.py 0 → 100644
View file @ b0ccdb11
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Definition of anchor labeler, which assigns ground truth boxes to anchors."""
import tensorflow as tf
class AnchorLabeler:
"""Labeler for dense object detector."""
def __init__(
self,
positive_class_weight=1.0,
positive_regression_weight=1.0,
negative_class_weight=1.0,
negative_regression_weight=0.0,
negative_class_label=-1,
ignore_class_label=-2,
negative_regression_label=0.,
ignore_regression_label=0.):
"""Constructs Anchor Labeler.
Args:
positive_class_weight: classification weight to be associated to positive
matched anchor. Defaults to 1.0.
positive_regression_weight: regression weight to be associated to positive
matched anchor. Defaults to 1.0.
negative_class_weight: classification weight to be associated to negative
matched anchor. Default to 1.0
negative_regression_weight: classification weight to be associated to
negative matched anchor. Default to 0.0.
negative_class_label: An integer for classification label to be associated
for negative matched anchor. Defaults to -1.
ignore_class_label: An integer for classification label to be associated
for ignored anchor. Defaults to -2.
negative_regression_label: A float for regression label to be associated
for negative matched anchor. Defaults to 0.
ignore_regression_label: A float for regression label to be associated
for ignored anchor. Defaults to 0.
"""
self.positive_class_weight = positive_class_weight
self.positive_regression_weight = positive_regression_weight
self.negative_class_weight = negative_class_weight
self.negative_regression_weight = negative_regression_weight
self.negative_class_label = negative_class_label
self.ignore_class_label = ignore_class_label
self.negative_regression_label = negative_regression_label
self.ignore_regression_label = ignore_regression_label
def __call__(self, boxes, labels, matches):
"""Labels anchors with ground truth inputs.
Args:
boxes: A float tensor with shape [N, 4] representing groundtruth boxes.
For each row, it stores [y0, x0, y1, x1] for four corners of a box.
labels: An integer tensor with shape [N, 1] representing groundtruth
classes.
matches: An integer tensor with shape [N] representing match results, must
be -1 for negative matched anchor, and -2 for ignored anchor.
Returns:
class_targets: A integer Tensor with shape [num_anchors].
box_targets: A float Tensor with shape [num_anchors, 4].
class_weights: A float Tensor with shape [num_anchors], that
serves as masking / sample weight for classification loss. Its value
is 1.0 for positive and negative matched anchors, and 0.0 for ignored
anchors.
box_weights: A float Tensor with shape [num_anchors], that
serves as masking / sample weight for regression loss. Its value is
1.0 for positive matched anchors, and 0.0 for negative and ignored
anchors.
"""
class_targets = self._gather_based_on_match(
matches, tf.cast(labels, tf.int32),
negative_value=tf.constant([self.negative_class_label], tf.int32),
ignored_value=tf.constant([self.ignore_class_label], tf.int32))
negative_reg_value = tf.constant(
[self.negative_regression_label] * 4, dtype=tf.float32)
ignore_reg_value = tf.constant(
[self.ignore_regression_label] * 4, dtype=tf.float32)
reg_targets = self._gather_based_on_match(
matches, boxes, negative_reg_value, ignore_reg_value)
num_gt_boxes = boxes.shape.as_list()[0] or tf.shape(boxes)[0]
groundtruth_class_weights = self.positive_class_weight * tf.ones(
[num_gt_boxes], dtype=tf.float32)
class_weights = self._gather_based_on_match(
matches, groundtruth_class_weights,
negative_value=self.negative_class_weight,
ignored_value=0.)
groundtruth_reg_weights = self.positive_regression_weight * tf.ones(
[num_gt_boxes], dtype=tf.float32)
reg_weights = self._gather_based_on_match(
matches, groundtruth_reg_weights,
negative_value=self.negative_regression_weight, ignored_value=0.)
return class_targets, reg_targets, class_weights, reg_weights
def _gather_based_on_match(
self, matches, inputs, negative_value, ignored_value):
"""Gathers elements from `input_tensor` based on match results.
For columns that are matched to a row, gathered_tensor[col] is set to
input_tensor[match[col]]. For columns that are unmatched,
gathered_tensor[col] is set to negative_value. Finally, for columns that
are ignored gathered_tensor[col] is set to ignored_value.
Note that the input_tensor.shape[1:] must match with unmatched_value.shape
and ignored_value.shape
Args:
matches: A integer tensor with shape [N] representing the
matching results of anchors. (1) match_results[i]>=0,
meaning that column i is matched with row match_results[i].
(2) match_results[i]=-1, meaning that column i is not matched.
(3) match_results[i]=-2, meaning that column i is ignored.
inputs: Tensor to gather values from.
negative_value: Constant tensor value for unmatched columns.
ignored_value: Constant tensor value for ignored columns.
Returns:
gathered_tensor: A tensor containing values gathered from input_tensor.
The shape of the gathered tensor is [match.shape[0]] +
input_tensor.shape[1:].
"""
inputs = tf.concat(
[tf.stack([ignored_value, negative_value]), inputs], axis=0)
gather_indices = tf.maximum(matches + 2, 0)
gathered_tensor = tf.gather(inputs, gather_indices)
return gathered_tensor
official/vision/keras_cv/ops/box_matcher.py 0 → 100644
View file @ b0ccdb11
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Box matcher implementation."""
import tensorflow as tf
class BoxMatcher:
"""Matcher based on highest value.
This class computes matches from a similarity matrix. Each column is matched
to a single row.
To support object detection target assignment this class enables setting both
positive_threshold (upper threshold) and negative_threshold (lower thresholds)
defining three categories of similarity which define whether examples are
positive, negative, or ignored:
(1) similarity >= positive_threshold: Highest similarity. Matched/Positive!
(2) positive_threshold > similarity >= negative_threshold: Medium similarity.
This is Ignored.
(3) negative_threshold > similarity: Lowest similarity for Negative Match.
For ignored matches this class sets the values in the Match object to -2.
"""
def __init__(
self,
positive_threshold,
negative_threshold=None,
force_match_for_each_row=False,
negative_value=-1,
ignore_value=-2):
"""Construct BoxMatcher.
Args:
positive_threshold: Threshold for positive matches. Positive if
sim >= positive_threshold, where sim is the maximum value of the
similarity matrix for a given column. Set to None for no threshold.
negative_threshold: Threshold for negative matches. Negative if
sim < negative_threshold or
positive_threshold > sim >= negative_threshold.
Defaults to positive_threshold when set to None.
force_match_for_each_row: If True, ensures that each row is matched to
at least one column (which is not guaranteed otherwise if the
positive_threshold is high). Defaults to False.
negative_value: An integer to fill for negative matches.
ignore_value: An integer to fill for ignored matches.
Raises:
ValueError: If negative_threshold > positive_threshold.
"""
self._positive_threshold = positive_threshold
if negative_threshold is None:
self._negative_threshold = positive_threshold
else:
if negative_threshold > positive_threshold:
raise ValueError('negative_threshold needs to be smaller or equal'
'to positive_threshold')
self._negative_threshold = negative_threshold
self._negative_value = negative_value
self._ignore_value = ignore_value
self._force_match_for_each_row = force_match_for_each_row
def __call__(self, similarity_matrix):
"""Tries to match each column of the similarity matrix to a row.
Args:
similarity_matrix: A float tensor of shape [N, M] representing any
similarity metric.
Returns:
A integer tensor with corresponding match indices for each of M columns,
for positive match, the match result will be the corresponding row index,
for negative match, the match will be `negative_value`, for ignored match,
the match result will be `ignore_value`.
"""
def _match_when_rows_are_empty():
"""Performs matching when the rows of similarity matrix are empty.
When the rows are empty, all detections are false positives. So we return
a tensor of -1's to indicate that the columns do not match to any rows.
Returns:
matches: int32 tensor indicating the row each column matches to.
"""
static_shape = similarity_matrix.shape.as_list()
num_cols = static_shape[1] or tf.shape(similarity_matrix)[1]
return -1 * tf.ones([num_cols], dtype=tf.int32)
def _match_when_rows_are_non_empty():
"""Performs matching when the rows of similarity matrix are non empty.
Returns:
matches: int32 tensor indicating the row each column matches to.
"""
# Matches for each column
matches = tf.argmax(input=similarity_matrix, axis=0, output_type=tf.int32)
# Deal with matched and unmatched threshold
if self._positive_threshold is not None:
# Get logical indices of ignored and unmatched columns as tf.int64
matched_vals = tf.reduce_max(similarity_matrix, axis=0)
below_negative_threshold = tf.greater(self._negative_threshold,
matched_vals)
between_thresholds = tf.logical_and(
tf.greater_equal(matched_vals, self._negative_threshold),
tf.greater(self._positive_threshold, matched_vals))
matches = self._set_values_using_indicator(matches,
below_negative_threshold,
self._negative_value)
matches = self._set_values_using_indicator(matches,
between_thresholds,
self._ignore_value)
if self._force_match_for_each_row:
num_gt_boxes = similarity_matrix.shape.as_list()[1] or tf.shape(
similarity_matrix)[1]
force_match_column_ids = tf.argmax(
input=similarity_matrix, axis=1, output_type=tf.int32)
force_match_column_indicators = tf.one_hot(
force_match_column_ids, depth=num_gt_boxes)
force_match_row_ids = tf.argmax(
input=force_match_column_indicators, axis=0, output_type=tf.int32)
force_match_column_mask = tf.cast(
tf.reduce_max(force_match_column_indicators, axis=0),
tf.bool)
final_matches = tf.where(force_match_column_mask, force_match_row_ids,
matches)
return final_matches
else:
return matches
num_gt_boxes = similarity_matrix.shape.as_list()[0] or tf.shape(
similarity_matrix)[0]
return tf.cond(
pred=tf.greater(num_gt_boxes, 0),
true_fn=_match_when_rows_are_non_empty,
false_fn=_match_when_rows_are_empty)
def _set_values_using_indicator(self, x, indicator, val):
"""Set the indicated fields of x to val.
Args:
x: tensor.
indicator: boolean with same shape as x.
val: scalar with value to set.
Returns:
modified tensor.
"""
indicator = tf.cast(indicator, x.dtype)
return tf.add(tf.multiply(x, 1 - indicator), val * indicator)
official/vision/keras_cv/ops/iou_similarity.py 0 → 100644
View file @ b0ccdb11
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Region Similarity Calculators."""
import tensorflow as tf
def area(box):
"""Computes area of boxes.
Args:
box: a float Tensor with [N, 4].
Returns:
a float tensor with [N].
"""
with tf.name_scope('Area'):
y_min, x_min, y_max, x_max = tf.split(
value=box, num_or_size_splits=4, axis=1)
return tf.squeeze((y_max - y_min) * (x_max - x_min), [1])
def intersection(box1, box2):
"""Compute pairwise intersection areas between boxes.
Args:
box1: a float Tensor with [N, 4].
box2: a float Tensor with [M, 4].
Returns:
a float tensor with shape [N, M] representing pairwise intersections
"""
with tf.name_scope('Intersection'):
y_min1, x_min1, y_max1, x_max1 = tf.split(
value=box1, num_or_size_splits=4, axis=1)
y_min2, x_min2, y_max2, x_max2 = tf.split(
value=box2, num_or_size_splits=4, axis=1)
y_min_max = tf.minimum(y_max1, tf.transpose(a=y_max2))
y_max_min = tf.maximum(y_min1, tf.transpose(a=y_min2))
intersect_heights = tf.maximum(0.0, y_min_max - y_max_min)
x_min_max = tf.minimum(x_max1, tf.transpose(a=x_max2))
x_max_min = tf.maximum(x_min1, tf.transpose(a=x_min2))
intersect_widths = tf.maximum(0.0, x_min_max - x_max_min)
return intersect_heights * intersect_widths
def iou(box1, box2):
"""Computes pairwise intersection-over-union between box collections.
Args:
box1: a float Tensor with [N, 4].
box2: a float Tensor with [M, 4].
Returns:
a tensor with shape [N, M] representing pairwise iou scores.
"""
intersections = intersection(box1, box2)
areas1 = area(box1)
areas2 = area(box2)
unions = (
tf.expand_dims(areas1, 1) + tf.expand_dims(areas2, 0) - intersections)
return tf.where(
tf.equal(intersections, 0.0), tf.zeros_like(intersections),
tf.truediv(intersections, unions))
class IouSimilarity():
"""Class to compute similarity based on Intersection over Union (IOU) metric.
"""
def __call__(self, groundtruth_boxes, anchors):
"""Compute pairwise IOU similarity between ground truth boxes and anchors.
Args:
groundtruth_boxes: a float Tensor with N boxes.
anchors: a float Tensor with M boxes.
Returns:
A tensor with shape [N, M] representing pairwise iou scores.
Input shape:
groundtruth_boxes: [N, 4]
anchors: [M, 4]
Output shape:
[N, M]
"""
with tf.name_scope('IOU'):
return iou(groundtruth_boxes, anchors)
official/vision/keras_cv/setup.py 0 → 100644
View file @ b0ccdb11
# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Setup script."""
import os
from setuptools import find_packages
from setuptools import setup
version = '0.0.1'
def _get_requirements():
"""Parses requirements.txt file."""
install_requires_tmp = []
dependency_links_tmp = []
with open(
os.path.join(os.path.dirname(__file__), './requirements.txt'), 'r') as f:
for line in f:
package_name = line.strip()
# Skip empty line or comments starting with "#".
if not package_name or package_name[0] == '#':
continue
if package_name.startswith('-e '):
dependency_links_tmp.append(package_name[3:].strip())
else:
install_requires_tmp.append(package_name)
return install_requires_tmp, dependency_links_tmp
install_requires, dependency_links = _get_requirements()
install_requires.append('tf-nightly')
install_requires.append('tensorflow-datasets')
setup(
name='keras-cv',
version=version,
description='Keras Computer Vision Library',
url='https://github.com/keras-team/keras-cv',
author='The Keras authors',
author_email='keras-team@google.com',
license='Apache License 2.0',
install_requires=install_requires,
classifiers=[
'Programming Language :: Python',
'Programming Language :: Python :: 3.6',
'Operating System :: Unix',
'Operating System :: Microsoft :: Windows',
'Operating System :: MacOS',
'Intended Audience :: Science/Research',
'Topic :: Scientific/Engineering',
'Topic :: Software Development'
],
packages=find_packages(exclude=('tests',)),
exclude_package_data={'': ['*_test.py',],},
dependency_links=dependency_links,
python_requires='>=3.6',
)
research/delf/README.md
View file @ b0ccdb11
......@@ -116,6 +116,11 @@ performance by ~4% mAP compared to ICCV'17 DELF model.
[R50-DELG](http://storage.googleapis.com/delf/r50delg_gld_20200814.tar.gz)).
Presented in the [DELG paper](https://arxiv.org/abs/2001.05027).
**DELG pre-trained on the Google-Landmarks dataset v2 (clean)**
([R101-DELG](https://storage.googleapis.com/delf/r101delg_gldv2clean_20200914.tar.gz),
[R50-DELG](https://storage.googleapis.com/delf/r50delg_gldv2clean_20200914.tar.gz)).
Presented in the [DELG paper](https://arxiv.org/abs/2001.05027).
**RN101-ArcFace pre-trained on the Google-Landmarks dataset v2 (train-clean)**
([link](https://storage.googleapis.com/delf/rn101_af_gldv2clean_20200814.tar.gz)).
Presented in the [GLDv2 paper](https://arxiv.org/abs/2004.01804).
......
research/delf/delf/python/delg/DELG_INSTRUCTIONS.md
View file @ b0ccdb11
......@@ -39,9 +39,9 @@ wget http://cmp.felk.cvut.cz/revisitop/data/datasets/rparis6k/gnd_rparis6k.mat
### Download model
This is necessary to reproduce the main paper results. This example shows the
R50-DELG model; the
[R101-DELG model](http://storage.googleapis.com/delf/r101delg_gld_20200814.tar.gz)
can be used as well, with similar steps.
R50-DELG model, pretrained on GLD; see the available pre-trained models
[here](../../../README.md#pre-trained-models), for other variants (eg, R101,
trained on GLDv2-clean).
```bash
# From models/research/delf/delf/python/delg
......@@ -54,11 +54,18 @@ tar -xvzf r50delg_gld_20200814.tar.gz
### Feature extraction
We present here commands for R50-DELG extraction on `roxford5k`.
We present here commands for R50-DELG (pretrained on GLD) extraction on
`roxford5k`.
- To use the R101-DELG model, first download it as done above for the R50
variant; then, replace the below argument `delf_config_path` by
- To use the R101-DELG model pretrained on GLD, first download it as mentioned
above; then, replace the below argument `delf_config_path` by
`r101delg_gld_config.pbtxt`
- To use the R50-DELG model pretrained on GLDv2-clean, first download it as
mentioned above; then, replace the below argument `delf_config_path` by
`r50delg_gldv2clean_config.pbtxt`
- To use the R101-DELG model pretrained on GLDv2-clean, first download it as
mentioned above; then, replace the below argument `delf_config_path` by
`r101delg_gldv2clean_config.pbtxt`
- To extract on `rparis6k` instead, please edit the arguments accordingly
(especially the `dataset_file_path` argument).
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment