Skip to content

GitLab

Unverified Commit 0cceabfc authored by Yiming Shi's avatar Yiming Shi Committed by GitHub
Browse files

Merge branch 'master' into move_to_keraslayers_fasterrcnn_fpn_keras_feature_extractor

parents 17821c0d 39ee0ac9
Hide whitespace changes
Inline Side-by-side
Showing with 98 additions and 3138 deletions
research/adv_imagenet_models/README.md deleted 100644 → 0
View file @ 17821c0d
![No Maintenance Intended](https://img.shields.io/badge/No%20Maintenance%20Intended-%E2%9C%95-red.svg)
![TensorFlow Requirement: 1.x](https://img.shields.io/badge/TensorFlow%20Requirement-1.x-brightgreen)
![TensorFlow 2 Not Supported](https://img.shields.io/badge/TensorFlow%202%20Not%20Supported-%E2%9C%95-red.svg)
# Adversarially trained ImageNet models
Pre-trained ImageNet models from the following papers:
* [Adversarial Machine Learning at Scale](https://arxiv.org/abs/1611.01236)
* [Ensemble Adversarial Training: Attacks and Defenses](https://arxiv.org/abs/1705.07204)
## Contact
Author: Alexey Kurakin,
github: [AlexeyKurakin](https://github.com/AlexeyKurakin)
## Pre-requesites and installation
Ensure that you have installed TensorFlow 1.1 or greater
([instructions](https://www.tensorflow.org/install/)).
You also need copy of ImageNet dataset if you want to run provided example.
Follow
[Preparing the dataset](https://github.com/tensorflow/models/tree/master/research/slim#Data)
instructions in TF-Slim library to get and preprocess ImageNet data.
## Available models
Following pre-trained models are available:
Network Architecture | Adversarial training | Checkpoint
---------------------|----------------------|----------------
Inception v3 | Step L.L. | [adv_inception_v3_2017_08_18.tar.gz](http://download.tensorflow.org/models/adv_inception_v3_2017_08_18.tar.gz)
Inception v3 | Step L.L. on ensemble of 3 models | [ens3_adv_inception_v3_2017_08_18.tar.gz](http://download.tensorflow.org/models/ens3_adv_inception_v3_2017_08_18.tar.gz)
Inception v3 | Step L.L. on ensemble of 4 models| [ens4_adv_inception_v3_2017_08_18.tar.gz](http://download.tensorflow.org/models/ens4_adv_inception_v3_2017_08_18.tar.gz)
Inception ResNet v2 | Step L.L. | [adv_inception_resnet_v2_2017_12_18.tar.gz](http://download.tensorflow.org/models/adv_inception_resnet_v2_2017_12_18.tar.gz)
Inception ResNet v2 | Step L.L. on ensemble of 3 models | [ens_adv_inception_resnet_v2_2017_08_18.tar.gz](http://download.tensorflow.org/models/ens_adv_inception_resnet_v2_2017_08_18.tar.gz)
All checkpoints are compatible with
[TF-Slim](https://github.com/tensorflow/models/tree/master/research/slim)
implementation of Inception v3 and Inception Resnet v2.
## How to evaluate models on ImageNet test data
Python script `eval_on_adversarial.py` allow you to evaluate provided models
on white-box adversarial examples generated from ImageNet test set.
Usage is following:
```bash
# ${MODEL_NAME} - type of network architecture,
# either "inception_v3" or "inception_resnet_v2"
# ${CHECKPOINT_PATH} - path to model checkpoint
# ${DATASET_DIR} - directory with ImageNet test set
# ${ADV_METHOD} - which method to use to generate adversarial images,
# supported method:
# "none" - use clean images from the dataset
# "stepll" - one step towards least likely class method (StepLL),
# see https://arxiv.org/abs/1611.01236 for details
# "stepllnoise" - RAND+StepLL method from https://arxiv.org/abs/1705.07204
# ${ADV_EPS} - size of adversarial perturbation, ignored when method is none
python eval_on_adversarial.py \
--model_name=${MODEL_NAME} \
--checkpoint_path=${CHECKPOINT_PATH} \
--dataset_dir=${DATASET_DIR} \
--batch_size=50 \
--adversarial_method=${ADV_METHOD} \
--adversarial_eps=${ADV_EPS}
```
Below is an example how to evaluate one of the models on RAND+StepLL adversarial
examples:
```bash
# Download checkpoint
CHECKPOINT_DIR=/tmp/checkpoints
mkdir ${CHECKPOINT_DIR}
wget http://download.tensorflow.org/models/ens_adv_inception_resnet_v2_2017_08_18.tar.gz
tar -xvf ens_adv_inception_resnet_v2_2017_08_18.tar.gz
mv ens_adv_inception_resnet_v2.ckpt* ${CHECKPOINT_DIR}
rm ens_adv_inception_resnet_v2_2017_08_18.tar.gz
# Run evaluation
python eval_on_adversarial.py \
--model_name=inception_v3 \
--checkpoint_path=${CHECKPOINT_DIR}/ens_adv_inception_resnet_v2.ckpt \
--dataset_dir=${DATASET_DIR} \
--batch_size=50 \
--adversarial_method=stepllnoise \
--adversarial_eps=16
```
research/adv_imagenet_models/eval_on_adversarial.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2017 The TensorFlow Authors All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Script which evaluates model on adversarial examples."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import math
import imagenet
import inception_resnet_v2
import tensorflow as tf
from tensorflow.contrib.slim.nets import inception
slim = tf.contrib.slim
tf.app.flags.DEFINE_integer(
'batch_size', 50, 'The number of samples in each batch.')
tf.app.flags.DEFINE_integer(
'max_num_batches', None,
'Max number of batches to evaluate by default use all.')
tf.app.flags.DEFINE_string(
'master', '', 'The address of the TensorFlow master to use.')
tf.app.flags.DEFINE_string(
'checkpoint_path', '/tmp/tfmodel/',
'The directory where the model was written to or an absolute path to a '
'checkpoint file.')
tf.app.flags.DEFINE_integer(
'num_preprocessing_threads', 4,
'The number of threads used to create the batches.')
tf.app.flags.DEFINE_string(
'split_name', 'validation', 'The name of the train/test split.')
tf.app.flags.DEFINE_string(
'dataset_dir', None, 'The directory where the dataset files are stored.')
tf.app.flags.DEFINE_string(
'model_name', 'inception_v3',
'Name of the model to use, either "inception_v3" or "inception_resnet_v2"')
tf.app.flags.DEFINE_float(
'moving_average_decay', None,
'The decay to use for the moving average.'
'If left as None, then moving averages are not used.')
tf.app.flags.DEFINE_string(
'adversarial_method', 'none',
'What kind of adversarial examples to use for evaluation. '
'Could be one of: "none", "stepll", "stepllnoise".')
tf.app.flags.DEFINE_float(
'adversarial_eps', 0.0,
'Size of adversarial perturbation in range [0, 255].')
FLAGS = tf.app.flags.FLAGS
IMAGE_SIZE = 299
NUM_CLASSES = 1001
def preprocess_for_eval(image, height, width,
central_fraction=0.875, scope=None):
"""Prepare one image for evaluation.
If height and width are specified it would output an image with that size by
applying resize_bilinear.
If central_fraction is specified it would crop the central fraction of the
input image.
Args:
image: 3-D Tensor of image. If dtype is tf.float32 then the range should be
[0, 1], otherwise it would converted to tf.float32 assuming that the range
is [0, MAX], where MAX is largest positive representable number for
int(8/16/32) data type (see `tf.image.convert_image_dtype` for details)
height: integer
width: integer
central_fraction: Optional Float, fraction of the image to crop.
scope: Optional scope for name_scope.
Returns:
3-D float Tensor of prepared image.
"""
with tf.name_scope(scope, 'eval_image', [image, height, width]):
if image.dtype != tf.float32:
image = tf.image.convert_image_dtype(image, dtype=tf.float32)
# Crop the central region of the image with an area containing 87.5% of
# the original image.
if central_fraction:
image = tf.image.central_crop(image, central_fraction=central_fraction)
if height and width:
# Resize the image to the specified height and width.
image = tf.expand_dims(image, 0)
image = tf.image.resize_bilinear(image, [height, width],
align_corners=False)
image = tf.squeeze(image, [0])
image = tf.subtract(image, 0.5)
image = tf.multiply(image, 2.0)
return image
def create_model(x, reuse=None):
"""Create model graph.
Args:
x: input images
reuse: reuse parameter which will be passed to underlying variable scopes.
Should be None first call and True every subsequent call.
Returns:
(logits, end_points) - tuple of model logits and enpoints
Raises:
ValueError: if model type specified by --model_name flag is invalid.
"""
if FLAGS.model_name == 'inception_v3':
with slim.arg_scope(inception.inception_v3_arg_scope()):
return inception.inception_v3(
x, num_classes=NUM_CLASSES, is_training=False, reuse=reuse)
elif FLAGS.model_name == 'inception_resnet_v2':
with slim.arg_scope(inception_resnet_v2.inception_resnet_v2_arg_scope()):
return inception_resnet_v2.inception_resnet_v2(
x, num_classes=NUM_CLASSES, is_training=False, reuse=reuse)
else:
raise ValueError('Invalid model name: %s' % (FLAGS.model_name))
def step_target_class_adversarial_images(x, eps, one_hot_target_class):
"""Base code for one step towards target class methods.
Args:
x: source images
eps: size of adversarial perturbation
one_hot_target_class: one hot encoded target classes for all images
Returns:
tensor with adversarial images
"""
logits, end_points = create_model(x, reuse=True)
cross_entropy = tf.losses.softmax_cross_entropy(one_hot_target_class,
logits,
label_smoothing=0.1,
weights=1.0)
cross_entropy += tf.losses.softmax_cross_entropy(one_hot_target_class,
end_points['AuxLogits'],
label_smoothing=0.1,
weights=0.4)
x_adv = x - eps * tf.sign(tf.gradients(cross_entropy, x)[0])
x_adv = tf.clip_by_value(x_adv, -1.0, 1.0)
return tf.stop_gradient(x_adv)
def stepll_adversarial_images(x, eps):
"""One step towards least likely class (Step L.L.) adversarial examples.
This method is an alternative to FGSM which does not use true classes.
Method is described in the "Adversarial Machine Learning at Scale" paper,
https://arxiv.org/abs/1611.01236
Args:
x: source images
eps: size of adversarial perturbation
Returns:
adversarial images
"""
logits, _ = create_model(x, reuse=True)
least_likely_class = tf.argmin(logits, 1)
one_hot_ll_class = tf.one_hot(least_likely_class, NUM_CLASSES)
return step_target_class_adversarial_images(x, eps, one_hot_ll_class)
def stepllnoise_adversarial_images(x, eps):
"""Step L.L. with noise method.
This is an imporvement of Step L.L. method. This method is better against
adversarially trained models which learn to mask gradient.
Method is described in the section "New randomized one shot attack" of
"Ensemble Adversarial Training: Attacks and Defenses" paper,
https://arxiv.org/abs/1705.07204
Args:
x: source images
eps: size of adversarial perturbation
Returns:
adversarial images
"""
logits, _ = create_model(x, reuse=True)
least_likely_class = tf.argmin(logits, 1)
one_hot_ll_class = tf.one_hot(least_likely_class, NUM_CLASSES)
x_noise = x + eps / 2 * tf.sign(tf.random_normal(x.shape))
return step_target_class_adversarial_images(x_noise, eps / 2,
one_hot_ll_class)
def get_input_images(dataset_images):
"""Gets input images for the evaluation.
Args:
dataset_images: tensor with dataset images
Returns:
tensor with input images, which is either dataset images or adversarial
images.
Raises:
ValueError: if adversarial method specified by --adversarial_method flag
is invalid.
"""
# adversarial_eps defines max difference of values of pixels if
# pixels are in range [0, 255]. However values of dataset pixels are
# in range [-1, 1], so converting epsilon.
eps = FLAGS.adversarial_eps / 255 * 2.0
if FLAGS.adversarial_method == 'stepll':
return stepll_adversarial_images(dataset_images, eps)
elif FLAGS.adversarial_method == 'stepllnoise':
return stepllnoise_adversarial_images(dataset_images, eps)
elif FLAGS.adversarial_method == 'none':
return dataset_images
else:
raise ValueError('Invalid adversarial method: %s'
% (FLAGS.adversarial_method))
def main(_):
if not FLAGS.dataset_dir:
raise ValueError('You must supply the dataset directory with --dataset_dir')
tf.logging.set_verbosity(tf.logging.INFO)
with tf.Graph().as_default():
tf_global_step = tf.train.get_or_create_global_step()
###################
# Prepare dataset #
###################
dataset = imagenet.get_split(FLAGS.split_name, FLAGS.dataset_dir)
provider = slim.dataset_data_provider.DatasetDataProvider(
dataset,
shuffle=False,
common_queue_capacity=2 * FLAGS.batch_size,
common_queue_min=FLAGS.batch_size)
[dataset_image, label] = provider.get(['image', 'label'])
dataset_image = preprocess_for_eval(dataset_image, IMAGE_SIZE, IMAGE_SIZE)
dataset_images, labels = tf.train.batch(
[dataset_image, label],
batch_size=FLAGS.batch_size,
num_threads=FLAGS.num_preprocessing_threads,
capacity=5 * FLAGS.batch_size)
########################################
# Define the model and input exampeles #
########################################
create_model(tf.placeholder(tf.float32, shape=dataset_images.shape))
input_images = get_input_images(dataset_images)
logits, _ = create_model(input_images, reuse=True)
if FLAGS.moving_average_decay > 0:
variable_averages = tf.train.ExponentialMovingAverage(
FLAGS.moving_average_decay, tf_global_step)
variables_to_restore = variable_averages.variables_to_restore(
slim.get_model_variables())
variables_to_restore[tf_global_step.op.name] = tf_global_step
else:
variables_to_restore = slim.get_variables_to_restore()
######################
# Define the metrics #
######################
predictions = tf.argmax(logits, 1)
labels = tf.squeeze(labels)
names_to_values, names_to_updates = slim.metrics.aggregate_metric_map({
'Accuracy': slim.metrics.streaming_accuracy(predictions, labels),
'Recall_5': slim.metrics.streaming_sparse_recall_at_k(
logits, tf.reshape(labels, [-1, 1]), 5),
})
######################
# Run evaluation #
######################
if FLAGS.max_num_batches:
num_batches = FLAGS.max_num_batches
else:
# This ensures that we make a single pass over all of the data.
num_batches = math.ceil(dataset.num_samples / float(FLAGS.batch_size))
if tf.gfile.IsDirectory(FLAGS.checkpoint_path):
checkpoint_path = tf.train.latest_checkpoint(FLAGS.checkpoint_path)
else:
checkpoint_path = FLAGS.checkpoint_path
tf.logging.info('Evaluating %s' % checkpoint_path)
top1_accuracy, top5_accuracy = slim.evaluation.evaluate_once(
master=FLAGS.master,
checkpoint_path=checkpoint_path,
logdir=None,
summary_op=None,
num_evals=num_batches,
eval_op=list(names_to_updates.values()),
final_op=[names_to_values['Accuracy'], names_to_values['Recall_5']],
variables_to_restore=variables_to_restore)
print('Top1 Accuracy: ', top1_accuracy)
print('Top5 Accuracy: ', top5_accuracy)
if __name__ == '__main__':
tf.app.run()
research/adv_imagenet_models/imagenet.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2017 The TensorFlow Authors All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Provides data for the ImageNet ILSVRC 2012 Dataset plus some bounding boxes.
Some images have one or more bounding boxes associated with the label of the
image. See details here: http://image-net.org/download-bboxes
WARNING: Don't use for object detection, in this case all the bounding boxes
of the image belong to just one class.
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import tensorflow as tf
slim = tf.contrib.slim
_FILE_PATTERN = '%s-*'
_SPLITS_TO_SIZES = {
'train': 1281167,
'validation': 50000,
}
_ITEMS_TO_DESCRIPTIONS = {
'image': 'A color image of varying height and width.',
'label': 'The label id of the image, integer between 0 and 999',
'label_text': 'The text of the label.',
'object/bbox': 'A list of bounding boxes.',
'object/label': 'A list of labels, one per each object.',
}
_NUM_CLASSES = 1001
def get_split(split_name, dataset_dir, file_pattern=None, reader=None):
"""Gets a dataset tuple with instructions for reading ImageNet.
Args:
split_name: A train/test split name.
dataset_dir: The base directory of the dataset sources.
file_pattern: The file pattern to use when matching the dataset sources.
It is assumed that the pattern contains a '%s' string so that the split
name can be inserted.
reader: The TensorFlow reader type.
Returns:
A `Dataset` namedtuple.
Raises:
ValueError: if `split_name` is not a valid train/test split.
"""
if split_name not in _SPLITS_TO_SIZES:
raise ValueError('split name %s was not recognized.' % split_name)
if not file_pattern:
file_pattern = _FILE_PATTERN
file_pattern = os.path.join(dataset_dir, file_pattern % split_name)
# Allowing None in the signature so that dataset_factory can use the default.
if reader is None:
reader = tf.TFRecordReader
keys_to_features = {
'image/encoded': tf.FixedLenFeature(
(), tf.string, default_value=''),
'image/format': tf.FixedLenFeature(
(), tf.string, default_value='jpeg'),
'image/class/label': tf.FixedLenFeature(
[], dtype=tf.int64, default_value=-1),
'image/class/text': tf.FixedLenFeature(
[], dtype=tf.string, default_value=''),
'image/object/bbox/xmin': tf.VarLenFeature(
dtype=tf.float32),
'image/object/bbox/ymin': tf.VarLenFeature(
dtype=tf.float32),
'image/object/bbox/xmax': tf.VarLenFeature(
dtype=tf.float32),
'image/object/bbox/ymax': tf.VarLenFeature(
dtype=tf.float32),
'image/object/class/label': tf.VarLenFeature(
dtype=tf.int64),
}
items_to_handlers = {
'image': slim.tfexample_decoder.Image('image/encoded', 'image/format'),
'label': slim.tfexample_decoder.Tensor('image/class/label'),
'label_text': slim.tfexample_decoder.Tensor('image/class/text'),
'object/bbox': slim.tfexample_decoder.BoundingBox(
['ymin', 'xmin', 'ymax', 'xmax'], 'image/object/bbox/'),
'object/label': slim.tfexample_decoder.Tensor('image/object/class/label'),
}
decoder = slim.tfexample_decoder.TFExampleDecoder(
keys_to_features, items_to_handlers)
return slim.dataset.Dataset(
data_sources=file_pattern,
reader=reader,
decoder=decoder,
num_samples=_SPLITS_TO_SIZES[split_name],
items_to_descriptions=_ITEMS_TO_DESCRIPTIONS,
num_classes=_NUM_CLASSES)
research/adv_imagenet_models/inception_resnet_v2.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2017 The TensorFlow Authors All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Contains the definition of the Inception Resnet V2 architecture.
As described in http://arxiv.org/abs/1602.07261.
Inception-v4, Inception-ResNet and the Impact of Residual Connections
on Learning
Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, Alex Alemi
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import tensorflow as tf
slim = tf.contrib.slim
def block35(net, scale=1.0, activation_fn=tf.nn.relu, scope=None, reuse=None):
"""Builds the 35x35 resnet block."""
with tf.variable_scope(scope, 'Block35', [net], reuse=reuse):
with tf.variable_scope('Branch_0'):
tower_conv = slim.conv2d(net, 32, 1, scope='Conv2d_1x1')
with tf.variable_scope('Branch_1'):
tower_conv1_0 = slim.conv2d(net, 32, 1, scope='Conv2d_0a_1x1')
tower_conv1_1 = slim.conv2d(tower_conv1_0, 32, 3, scope='Conv2d_0b_3x3')
with tf.variable_scope('Branch_2'):
tower_conv2_0 = slim.conv2d(net, 32, 1, scope='Conv2d_0a_1x1')
tower_conv2_1 = slim.conv2d(tower_conv2_0, 48, 3, scope='Conv2d_0b_3x3')
tower_conv2_2 = slim.conv2d(tower_conv2_1, 64, 3, scope='Conv2d_0c_3x3')
mixed = tf.concat(axis=3, values=[tower_conv, tower_conv1_1, tower_conv2_2])
up = slim.conv2d(mixed, net.get_shape()[3], 1, normalizer_fn=None,
activation_fn=None, scope='Conv2d_1x1')
net += scale * up
if activation_fn:
net = activation_fn(net)
return net
def block17(net, scale=1.0, activation_fn=tf.nn.relu, scope=None, reuse=None):
"""Builds the 17x17 resnet block."""
with tf.variable_scope(scope, 'Block17', [net], reuse=reuse):
with tf.variable_scope('Branch_0'):
tower_conv = slim.conv2d(net, 192, 1, scope='Conv2d_1x1')
with tf.variable_scope('Branch_1'):
tower_conv1_0 = slim.conv2d(net, 128, 1, scope='Conv2d_0a_1x1')
tower_conv1_1 = slim.conv2d(tower_conv1_0, 160, [1, 7],
scope='Conv2d_0b_1x7')
tower_conv1_2 = slim.conv2d(tower_conv1_1, 192, [7, 1],
scope='Conv2d_0c_7x1')
mixed = tf.concat(axis=3, values=[tower_conv, tower_conv1_2])
up = slim.conv2d(mixed, net.get_shape()[3], 1, normalizer_fn=None,
activation_fn=None, scope='Conv2d_1x1')
net += scale * up
if activation_fn:
net = activation_fn(net)
return net
def block8(net, scale=1.0, activation_fn=tf.nn.relu, scope=None, reuse=None):
"""Builds the 8x8 resnet block."""
with tf.variable_scope(scope, 'Block8', [net], reuse=reuse):
with tf.variable_scope('Branch_0'):
tower_conv = slim.conv2d(net, 192, 1, scope='Conv2d_1x1')
with tf.variable_scope('Branch_1'):
tower_conv1_0 = slim.conv2d(net, 192, 1, scope='Conv2d_0a_1x1')
tower_conv1_1 = slim.conv2d(tower_conv1_0, 224, [1, 3],
scope='Conv2d_0b_1x3')
tower_conv1_2 = slim.conv2d(tower_conv1_1, 256, [3, 1],
scope='Conv2d_0c_3x1')
mixed = tf.concat(axis=3, values=[tower_conv, tower_conv1_2])
up = slim.conv2d(mixed, net.get_shape()[3], 1, normalizer_fn=None,
activation_fn=None, scope='Conv2d_1x1')
net += scale * up
if activation_fn:
net = activation_fn(net)
return net
def inception_resnet_v2_base(inputs,
final_endpoint='Conv2d_7b_1x1',
output_stride=16,
align_feature_maps=False,
scope=None):
"""Inception model from http://arxiv.org/abs/1602.07261.
Constructs an Inception Resnet v2 network from inputs to the given final
endpoint. This method can construct the network up to the final inception
block Conv2d_7b_1x1.
Args:
inputs: a tensor of size [batch_size, height, width, channels].
final_endpoint: specifies the endpoint to construct the network up to. It
can be one of ['Conv2d_1a_3x3', 'Conv2d_2a_3x3', 'Conv2d_2b_3x3',
'MaxPool_3a_3x3', 'Conv2d_3b_1x1', 'Conv2d_4a_3x3', 'MaxPool_5a_3x3',
'Mixed_5b', 'Mixed_6a', 'PreAuxLogits', 'Mixed_7a', 'Conv2d_7b_1x1']
output_stride: A scalar that specifies the requested ratio of input to
output spatial resolution. Only supports 8 and 16.
align_feature_maps: When true, changes all the VALID paddings in the network
to SAME padding so that the feature maps are aligned.
scope: Optional variable_scope.
Returns:
tensor_out: output tensor corresponding to the final_endpoint.
end_points: a set of activations for external use, for example summaries or
losses.
Raises:
ValueError: if final_endpoint is not set to one of the predefined values,
or if the output_stride is not 8 or 16, or if the output_stride is 8 and
we request an end point after 'PreAuxLogits'.
"""
if output_stride != 8 and output_stride != 16:
raise ValueError('output_stride must be 8 or 16.')
padding = 'SAME' if align_feature_maps else 'VALID'
end_points = {}
def add_and_check_final(name, net):
end_points[name] = net
return name == final_endpoint
with tf.variable_scope(scope, 'InceptionResnetV2', [inputs]):
with slim.arg_scope([slim.conv2d, slim.max_pool2d, slim.avg_pool2d],
stride=1, padding='SAME'):
# 149 x 149 x 32
net = slim.conv2d(inputs, 32, 3, stride=2, padding=padding,
scope='Conv2d_1a_3x3')
if add_and_check_final('Conv2d_1a_3x3', net): return net, end_points
# 147 x 147 x 32
net = slim.conv2d(net, 32, 3, padding=padding,
scope='Conv2d_2a_3x3')
if add_and_check_final('Conv2d_2a_3x3', net): return net, end_points
# 147 x 147 x 64
net = slim.conv2d(net, 64, 3, scope='Conv2d_2b_3x3')
if add_and_check_final('Conv2d_2b_3x3', net): return net, end_points
# 73 x 73 x 64
net = slim.max_pool2d(net, 3, stride=2, padding=padding,
scope='MaxPool_3a_3x3')
if add_and_check_final('MaxPool_3a_3x3', net): return net, end_points
# 73 x 73 x 80
net = slim.conv2d(net, 80, 1, padding=padding,
scope='Conv2d_3b_1x1')
if add_and_check_final('Conv2d_3b_1x1', net): return net, end_points
# 71 x 71 x 192
net = slim.conv2d(net, 192, 3, padding=padding,
scope='Conv2d_4a_3x3')
if add_and_check_final('Conv2d_4a_3x3', net): return net, end_points
# 35 x 35 x 192
net = slim.max_pool2d(net, 3, stride=2, padding=padding,
scope='MaxPool_5a_3x3')
if add_and_check_final('MaxPool_5a_3x3', net): return net, end_points
# 35 x 35 x 320
with tf.variable_scope('Mixed_5b'):
with tf.variable_scope('Branch_0'):
tower_conv = slim.conv2d(net, 96, 1, scope='Conv2d_1x1')
with tf.variable_scope('Branch_1'):
tower_conv1_0 = slim.conv2d(net, 48, 1, scope='Conv2d_0a_1x1')
tower_conv1_1 = slim.conv2d(tower_conv1_0, 64, 5,
scope='Conv2d_0b_5x5')
with tf.variable_scope('Branch_2'):
tower_conv2_0 = slim.conv2d(net, 64, 1, scope='Conv2d_0a_1x1')
tower_conv2_1 = slim.conv2d(tower_conv2_0, 96, 3,
scope='Conv2d_0b_3x3')
tower_conv2_2 = slim.conv2d(tower_conv2_1, 96, 3,
scope='Conv2d_0c_3x3')
with tf.variable_scope('Branch_3'):
tower_pool = slim.avg_pool2d(net, 3, stride=1, padding='SAME',
scope='AvgPool_0a_3x3')
tower_pool_1 = slim.conv2d(tower_pool, 64, 1,
scope='Conv2d_0b_1x1')
net = tf.concat(
[tower_conv, tower_conv1_1, tower_conv2_2, tower_pool_1], 3)
if add_and_check_final('Mixed_5b', net): return net, end_points
# TODO(alemi): Register intermediate endpoints
net = slim.repeat(net, 10, block35, scale=0.17)
# 17 x 17 x 1088 if output_stride == 8,
# 33 x 33 x 1088 if output_stride == 16
use_atrous = output_stride == 8
with tf.variable_scope('Mixed_6a'):
with tf.variable_scope('Branch_0'):
tower_conv = slim.conv2d(net, 384, 3, stride=1 if use_atrous else 2,
padding=padding,
scope='Conv2d_1a_3x3')
with tf.variable_scope('Branch_1'):
tower_conv1_0 = slim.conv2d(net, 256, 1, scope='Conv2d_0a_1x1')
tower_conv1_1 = slim.conv2d(tower_conv1_0, 256, 3,
scope='Conv2d_0b_3x3')
tower_conv1_2 = slim.conv2d(tower_conv1_1, 384, 3,
stride=1 if use_atrous else 2,
padding=padding,
scope='Conv2d_1a_3x3')
with tf.variable_scope('Branch_2'):
tower_pool = slim.max_pool2d(net, 3, stride=1 if use_atrous else 2,
padding=padding,
scope='MaxPool_1a_3x3')
net = tf.concat([tower_conv, tower_conv1_2, tower_pool], 3)
if add_and_check_final('Mixed_6a', net): return net, end_points
# TODO(alemi): register intermediate endpoints
with slim.arg_scope([slim.conv2d], rate=2 if use_atrous else 1):
net = slim.repeat(net, 20, block17, scale=0.10)
if add_and_check_final('PreAuxLogits', net): return net, end_points
if output_stride == 8:
# TODO(gpapan): Properly support output_stride for the rest of the net.
raise ValueError('output_stride==8 is only supported up to the '
'PreAuxlogits end_point for now.')
# 8 x 8 x 2080
with tf.variable_scope('Mixed_7a'):
with tf.variable_scope('Branch_0'):
tower_conv = slim.conv2d(net, 256, 1, scope='Conv2d_0a_1x1')
tower_conv_1 = slim.conv2d(tower_conv, 384, 3, stride=2,
padding=padding,
scope='Conv2d_1a_3x3')
with tf.variable_scope('Branch_1'):
tower_conv1 = slim.conv2d(net, 256, 1, scope='Conv2d_0a_1x1')
tower_conv1_1 = slim.conv2d(tower_conv1, 288, 3, stride=2,
padding=padding,
scope='Conv2d_1a_3x3')
with tf.variable_scope('Branch_2'):
tower_conv2 = slim.conv2d(net, 256, 1, scope='Conv2d_0a_1x1')
tower_conv2_1 = slim.conv2d(tower_conv2, 288, 3,
scope='Conv2d_0b_3x3')
tower_conv2_2 = slim.conv2d(tower_conv2_1, 320, 3, stride=2,
padding=padding,
scope='Conv2d_1a_3x3')
with tf.variable_scope('Branch_3'):
tower_pool = slim.max_pool2d(net, 3, stride=2,
padding=padding,
scope='MaxPool_1a_3x3')
net = tf.concat(
[tower_conv_1, tower_conv1_1, tower_conv2_2, tower_pool], 3)
if add_and_check_final('Mixed_7a', net): return net, end_points
# TODO(alemi): register intermediate endpoints
net = slim.repeat(net, 9, block8, scale=0.20)
net = block8(net, activation_fn=None)
# 8 x 8 x 1536
net = slim.conv2d(net, 1536, 1, scope='Conv2d_7b_1x1')
if add_and_check_final('Conv2d_7b_1x1', net): return net, end_points
raise ValueError('final_endpoint (%s) not recognized', final_endpoint)
def inception_resnet_v2(inputs, num_classes=1001, is_training=True,
dropout_keep_prob=0.8,
reuse=None,
scope='InceptionResnetV2',
create_aux_logits=True):
"""Creates the Inception Resnet V2 model.
Args:
inputs: a 4-D tensor of size [batch_size, height, width, 3].
num_classes: number of predicted classes.
is_training: whether is training or not.
dropout_keep_prob: float, the fraction to keep before final layer.
reuse: whether or not the network and its variables should be reused. To be
able to reuse 'scope' must be given.
scope: Optional variable_scope.
create_aux_logits: Whether to include the auxilliary logits.
Returns:
logits: the logits outputs of the model.
end_points: the set of end_points from the inception model.
"""
end_points = {}
with tf.variable_scope(scope, 'InceptionResnetV2', [inputs, num_classes],
reuse=reuse) as scope:
with slim.arg_scope([slim.batch_norm, slim.dropout],
is_training=is_training):
net, end_points = inception_resnet_v2_base(inputs, scope=scope)
if create_aux_logits:
with tf.variable_scope('AuxLogits'):
aux = end_points['PreAuxLogits']
aux = slim.avg_pool2d(aux, 5, stride=3, padding='VALID',
scope='Conv2d_1a_3x3')
aux = slim.conv2d(aux, 128, 1, scope='Conv2d_1b_1x1')
aux = slim.conv2d(aux, 768, aux.get_shape()[1:3],
padding='VALID', scope='Conv2d_2a_5x5')
aux = slim.flatten(aux)
aux = slim.fully_connected(aux, num_classes, activation_fn=None,
scope='Logits')
end_points['AuxLogits'] = aux
with tf.variable_scope('Logits'):
net = slim.avg_pool2d(net, net.get_shape()[1:3], padding='VALID',
scope='AvgPool_1a_8x8')
net = slim.flatten(net)
net = slim.dropout(net, dropout_keep_prob, is_training=is_training,
scope='Dropout')
end_points['PreLogitsFlatten'] = net
logits = slim.fully_connected(net, num_classes, activation_fn=None,
scope='Logits')
end_points['Logits'] = logits
end_points['Predictions'] = tf.nn.softmax(logits, name='Predictions')
return logits, end_points
inception_resnet_v2.default_image_size = 299
def inception_resnet_v2_arg_scope(weight_decay=0.00004,
batch_norm_decay=0.9997,
batch_norm_epsilon=0.001):
"""Returns the scope with the default parameters for inception_resnet_v2.
Args:
weight_decay: the weight decay for weights variables.
batch_norm_decay: decay for the moving average of batch_norm momentums.
batch_norm_epsilon: small float added to variance to avoid dividing by zero.
Returns:
a arg_scope with the parameters needed for inception_resnet_v2.
"""
# Set weight_decay for weights in conv2d and fully_connected layers.
with slim.arg_scope([slim.conv2d, slim.fully_connected],
weights_regularizer=slim.l2_regularizer(weight_decay),
biases_regularizer=slim.l2_regularizer(weight_decay)):
batch_norm_params = {
'decay': batch_norm_decay,
'epsilon': batch_norm_epsilon,
}
# Set activation_fn and parameters for batch_norm.
with slim.arg_scope([slim.conv2d], activation_fn=tf.nn.relu,
normalizer_fn=slim.batch_norm,
normalizer_params=batch_norm_params) as scope:
return scope
research/adversarial_crypto/README.md deleted 100644 → 0
View file @ 17821c0d
![No Maintenance Intended](https://img.shields.io/badge/No%20Maintenance%20Intended-%E2%9C%95-red.svg)
![TensorFlow Requirement: 1.x](https://img.shields.io/badge/TensorFlow%20Requirement-1.x-brightgreen)
![TensorFlow 2 Not Supported](https://img.shields.io/badge/TensorFlow%202%20Not%20Supported-%E2%9C%95-red.svg)
# Learning to Protect Communications with Adversarial Neural Cryptography
This is a slightly-updated model used for the paper
["Learning to Protect Communications with Adversarial Neural
Cryptography"](https://arxiv.org/abs/1610.06918).
> We ask whether neural networks can learn to use secret keys to protect
> information from other neural networks. Specifically, we focus on ensuring
> confidentiality properties in a multiagent system, and we specify those
> properties in terms of an adversary. Thus, a system may consist of neural
> networks named Alice and Bob, and we aim to limit what a third neural
> network named Eve learns from eavesdropping on the communication between
> Alice and Bob. We do not prescribe specific cryptographic algorithms to
> these neural networks; instead, we train end-to-end, adversarially.
> We demonstrate that the neural networks can learn how to perform forms of
> encryption and decryption, and also how to apply these operations
> selectively in order to meet confidentiality goals.
This code allows you to train encoder/decoder/adversary network triplets
and evaluate their effectiveness on randomly generated input and key
pairs.
## Prerequisites
The only software requirements for running the encoder and decoder is having
TensorFlow installed.
Requires TensorFlow r0.12 or later.
## Training and evaluating
After installing TensorFlow and ensuring that your paths are configured
appropriately:
```
python train_eval.py
```
This will begin training a fresh model. If and when the model becomes
sufficiently well-trained, it will reset the Eve model multiple times
and retrain it from scratch, outputting the accuracy thus obtained
in each run.
## Model differences from the paper
The model has been simplified slightly from the one described in
the paper - the convolutional layer width was reduced by a factor
of two. In the version in the paper, there was a nonlinear unit
after the fully-connected layer; that nonlinear has been removed
here. These changes improve the robustness of training. The
initializer for the convolution layers has switched to the
`tf.contrib.layers default` of `xavier_initializer` instead of
a simpler `truncated_normal`.
## Contact information
This model repository is maintained by David G. Andersen
([dave-andersen](https://github.com/dave-andersen)).
research/adversarial_crypto/train_eval.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2016 The TensorFlow Authors All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Adversarial training to learn trivial encryption functions,
from the paper "Learning to Protect Communications with
Adversarial Neural Cryptography", Abadi & Andersen, 2016.
https://arxiv.org/abs/1610.06918
This program creates and trains three neural networks,
termed Alice, Bob, and Eve. Alice takes inputs
in_m (message), in_k (key) and outputs 'ciphertext'.
Bob takes inputs in_k, ciphertext and tries to reconstruct
the message.
Eve is an adversarial network that takes input ciphertext
and also tries to reconstruct the message.
The main function attempts to train these networks and then
evaluates them, all on random plaintext and key values.
"""
# TensorFlow Python 3 compatibility
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import signal
import sys
from six.moves import xrange # pylint: disable=redefined-builtin
import tensorflow as tf
flags = tf.app.flags
flags.DEFINE_float('learning_rate', 0.0008, 'Constant learning rate')
flags.DEFINE_integer('batch_size', 4096, 'Batch size')
FLAGS = flags.FLAGS
# Input and output configuration.
TEXT_SIZE = 16
KEY_SIZE = 16
# Training parameters.
ITERS_PER_ACTOR = 1
EVE_MULTIPLIER = 2 # Train Eve 2x for every step of Alice/Bob
# Train until either max loops or Alice/Bob "good enough":
MAX_TRAINING_LOOPS = 850000
BOB_LOSS_THRESH = 0.02 # Exit when Bob loss < 0.02 and Eve > 7.7 bits
EVE_LOSS_THRESH = 7.7
# Logging and evaluation.
PRINT_EVERY = 200 # In training, log every 200 steps.
EVE_EXTRA_ROUNDS = 2000 # At end, train eve a bit more.
RETRAIN_EVE_ITERS = 10000 # Retrain eve up to ITERS*LOOPS times.
RETRAIN_EVE_LOOPS = 25 # With an evaluation each loop
NUMBER_OF_EVE_RESETS = 5 # And do this up to 5 times with a fresh eve.
# Use EVAL_BATCHES samples each time we check accuracy.
EVAL_BATCHES = 1
def batch_of_random_bools(batch_size, n):
"""Return a batch of random "boolean" numbers.
Args:
batch_size: Batch size dimension of returned tensor.
n: number of entries per batch.
Returns:
A [batch_size, n] tensor of "boolean" numbers, where each number is
preresented as -1 or 1.
"""
as_int = tf.random.uniform(
[batch_size, n], minval=0, maxval=2, dtype=tf.int32)
expanded_range = (as_int * 2) - 1
return tf.cast(expanded_range, tf.float32)
class AdversarialCrypto(object):
"""Primary model implementation class for Adversarial Neural Crypto.
This class contains the code for the model itself,
and when created, plumbs the pathways from Alice to Bob and
Eve, creates the optimizers and loss functions, etc.
Attributes:
eve_loss: Eve's loss function.
bob_loss: Bob's loss function. Different units from eve_loss.
eve_optimizer: A tf op that runs Eve's optimizer.
bob_optimizer: A tf op that runs Bob's optimizer.
bob_reconstruction_loss: Bob's message reconstruction loss,
which is comparable to eve_loss.
reset_eve_vars: Execute this op to completely reset Eve.
"""
def get_message_and_key(self):
"""Generate random pseudo-boolean key and message values."""
batch_size = tf.compat.v1.placeholder_with_default(FLAGS.batch_size, shape=[])
in_m = batch_of_random_bools(batch_size, TEXT_SIZE)
in_k = batch_of_random_bools(batch_size, KEY_SIZE)
return in_m, in_k
def model(self, collection, message, key=None):
"""The model for Alice, Bob, and Eve. If key=None, the first fully connected layer
takes only the message as inputs. Otherwise, it uses both the key
and the message.
Args:
collection: The graph keys collection to add new vars to.
message: The input message to process.
key: The input key (if any) to use.
"""
if key is not None:
combined_message = tf.concat(axis=1, values=[message, key])
else:
combined_message = message
# Ensure that all variables created are in the specified collection.
with tf.contrib.framework.arg_scope(
[tf.contrib.layers.fully_connected, tf.contrib.layers.conv2d],
variables_collections=[collection]):
fc = tf.contrib.layers.fully_connected(
combined_message,
TEXT_SIZE + KEY_SIZE,
biases_initializer=tf.constant_initializer(0.0),
activation_fn=None)
# Perform a sequence of 1D convolutions (by expanding the message out to 2D
# and then squeezing it back down).
fc = tf.expand_dims(fc, 2) # 2D
fc = tf.expand_dims(fc, 3) # 3D -- conv2d needs a depth
# 2,1 -> 1,2
conv = tf.contrib.layers.conv2d(
fc, 2, 2, 2, 'SAME', activation_fn=tf.nn.sigmoid)
# 1,2 -> 1, 2
conv = tf.contrib.layers.conv2d(
conv, 2, 1, 1, 'SAME', activation_fn=tf.nn.sigmoid)
# 1,2 -> 1, 1
conv = tf.contrib.layers.conv2d(
conv, 1, 1, 1, 'SAME', activation_fn=tf.nn.tanh)
conv = tf.squeeze(conv, 3)
conv = tf.squeeze(conv, 2)
return conv
def __init__(self):
in_m, in_k = self.get_message_and_key()
encrypted = self.model('alice', in_m, in_k)
decrypted = self.model('bob', encrypted, in_k)
eve_out = self.model('eve', encrypted, None)
self.reset_eve_vars = tf.group(
*[w.initializer for w in tf.compat.v1.get_collection('eve')])
optimizer = tf.compat.v1.train.AdamOptimizer(learning_rate=FLAGS.learning_rate)
# Eve's goal is to decrypt the entire message:
eve_bits_wrong = tf.reduce_sum(
tf.abs((eve_out + 1.0) / 2.0 - (in_m + 1.0) / 2.0), [1])
self.eve_loss = tf.reduce_sum(eve_bits_wrong)
self.eve_optimizer = optimizer.minimize(
self.eve_loss, var_list=tf.compat.v1.get_collection('eve'))
# Alice and Bob want to be accurate...
self.bob_bits_wrong = tf.reduce_sum(
tf.abs((decrypted + 1.0) / 2.0 - (in_m + 1.0) / 2.0), [1])
# ... and to not let Eve do better than guessing.
self.bob_reconstruction_loss = tf.reduce_sum(self.bob_bits_wrong)
bob_eve_error_deviation = tf.abs(float(TEXT_SIZE) / 2.0 - eve_bits_wrong)
# 7-9 bits wrong is OK too, so we squish the error function a bit.
# Without doing this, we often tend to hang out at 0.25 / 7.5 error,
# and it seems bad to have continued, high communication error.
bob_eve_loss = tf.reduce_sum(
tf.square(bob_eve_error_deviation) / (TEXT_SIZE / 2)**2)
# Rescale the losses to [0, 1] per example and combine.
self.bob_loss = (self.bob_reconstruction_loss / TEXT_SIZE + bob_eve_loss)
self.bob_optimizer = optimizer.minimize(
self.bob_loss,
var_list=(tf.compat.v1.get_collection('alice') + tf.compat.v1.get_collection('bob')))
def doeval(s, ac, n, itercount):
"""Evaluate the current network on n batches of random examples.
Args:
s: The current TensorFlow session
ac: an instance of the AdversarialCrypto class
n: The number of iterations to run.
itercount: Iteration count label for logging.
Returns:
Bob and Eve's loss, as a percent of bits incorrect.
"""
bob_loss_accum = 0
eve_loss_accum = 0
for _ in xrange(n):
bl, el = s.run([ac.bob_reconstruction_loss, ac.eve_loss])
bob_loss_accum += bl
eve_loss_accum += el
bob_loss_percent = bob_loss_accum / (n * FLAGS.batch_size)
eve_loss_percent = eve_loss_accum / (n * FLAGS.batch_size)
print('%10d\t%20.2f\t%20.2f'%(itercount, bob_loss_percent, eve_loss_percent))
sys.stdout.flush()
return bob_loss_percent, eve_loss_percent
def train_until_thresh(s, ac):
for j in xrange(MAX_TRAINING_LOOPS):
for _ in xrange(ITERS_PER_ACTOR):
s.run(ac.bob_optimizer)
for _ in xrange(ITERS_PER_ACTOR * EVE_MULTIPLIER):
s.run(ac.eve_optimizer)
if j % PRINT_EVERY == 0:
bob_avg_loss, eve_avg_loss = doeval(s, ac, EVAL_BATCHES, j)
if (bob_avg_loss < BOB_LOSS_THRESH and eve_avg_loss > EVE_LOSS_THRESH):
print('Target losses achieved.')
return True
return False
def train_and_evaluate():
"""Run the full training and evaluation loop."""
ac = AdversarialCrypto()
init = tf.compat.v1.global_variables_initializer()
with tf.compat.v1.Session() as s:
s.run(init)
print('# Batch size: ', FLAGS.batch_size)
print('# %10s\t%20s\t%20s'%("Iter","Bob_Recon_Error","Eve_Recon_Error"))
if train_until_thresh(s, ac):
for _ in xrange(EVE_EXTRA_ROUNDS):
s.run(ac.eve_optimizer)
print('Loss after eve extra training:')
doeval(s, ac, EVAL_BATCHES * 2, 0)
for _ in xrange(NUMBER_OF_EVE_RESETS):
print('Resetting Eve')
s.run(ac.reset_eve_vars)
eve_counter = 0
for _ in xrange(RETRAIN_EVE_LOOPS):
for _ in xrange(RETRAIN_EVE_ITERS):
eve_counter += 1
s.run(ac.eve_optimizer)
doeval(s, ac, EVAL_BATCHES, eve_counter)
doeval(s, ac, EVAL_BATCHES, eve_counter)
def main(unused_argv):
# Exit more quietly with Ctrl-C.
signal.signal(signal.SIGINT, signal.SIG_DFL)
train_and_evaluate()
if __name__ == '__main__':
tf.compat.v1.app.run()
research/adversarial_logit_pairing/README.md deleted 100644 → 0
View file @ 17821c0d
![No Maintenance Intended](https://img.shields.io/badge/No%20Maintenance%20Intended-%E2%9C%95-red.svg)
![TensorFlow Requirement: 1.x](https://img.shields.io/badge/TensorFlow%20Requirement-1.x-brightgreen)
![TensorFlow 2 Not Supported](https://img.shields.io/badge/TensorFlow%202%20Not%20Supported-%E2%9C%95-red.svg)
# Adversarial logit pairing
This directory contains implementation of
[Adversarial logit pairing](https://arxiv.org/abs/1803.06373) paper as well as
few models pre-trained on ImageNet and Tiny ImageNet.
Please contact [Alexey Kurakin](https://github.com/AlexeyKurakin) regarding
this code.
## Pre-requesites
Code dependencies:
* TensorFlow 1.8 and Python 2.7 (other versions may work, but were not tested)
* [Abseil Python](https://github.com/abseil/abseil-py).
* Script which converts Tiny Imagenet dataset into TFRecord format also
depends on [Pandas](https://pandas.pydata.org/).
## Datasets
To use this code you need to download datasets. You only need to download
those datasets which you're going to use. Following list of datasets is
supported:
* [ImageNet](http://www.image-net.org/). Follow
[Preparing the datasets](https://github.com/tensorflow/models/tree/master/research/slim#Data)
instructions in TF-Slim documentation to download and convert ImageNet dataset
to TFRecord format.
* [Tiny ImageNet](https://tiny-imagenet.herokuapp.com/).
To obtain Tiny ImageNet dataset do following:
```
# Download zip archive with TinyImagenet
curl -O http://cs231n.stanford.edu/tiny-imagenet-200.zip
# Extract archive
unzip tiny-imagenet-200.zip
# Convert dataset to TFRecord format
mkdir tiny-imagenet-tfrecord
python tiny_imagenet_converter/converter.py \
--input_dir=tiny-imagenet-200 \
--output_dir=tiny-imagenet-tfrecord
```
## Running the code
NOTE: Provided code supports distributed training on multiple machines,
and all provided checkpoints were trained in a distributed way. However it is
beyond the scope of this document to describe how to do distributed training.
Readed should refer to
[other material](https://www.tensorflow.org/deploy/distributed) to learn
about it.
### Training
Following command runs training:
```
# Following arguments has to be specified for training:
# - MAX_NUMBER_OF_TRAINING_STEPS - maximum number of training steps,
# omit this flag or set it to -1 to have unlimited number of training steps.
# - MODEL_NAME - name of the model, now only "resnet_v2_50" is supported.
# - MOVING_AVG_DECAY - decay rate for exponential moving average of the
# trainable variables. Training with exponential moving average usually
# leads to better accuracy. Default of 0.9999. -1 disable exponential moving
# average. Default works well, so typically you set it only if you want
# to disable this feature.
# - HYPERPARAMETERS - string with hyperparameters,
# see model_lib.py for full list of hyperparameters.
# - DATASET - dataset, either "imagenet" or "tiny_imagenet".
# - IMAGE_SIZE - size of the image (single number).
# - OUTPUT_DIRECTORY - directory where to write results.
# - IMAGENET_DIR - directory with ImageNet dataset in TFRecord format.
# - TINY_IMAGENET_DIR - directory with Tiny ImageNet dataset in TFRecord format.
#
# Note that only one of IMAGENET_DIR or TINY_IMAGENET_DIR has to be provided
# depending on which dataset you use.
#
python train.py \
--max_steps="${MAX_NUMBER_OF_TRAINING_STEPS}" \
--model_name="${MODEL_NAME}" \
--moving_average_decay="${MOVING_AVG_DECAY}" \
--hparams="${HYPERPARAMETERS}" \
--dataset="${DATASET}" \
--dataset_image_size="${IMAGE_SIZE}" \
--output_dir="${OUTPUT_DIRECTORY}" \
--imagenet_data_dir="${IMAGENET_DIR}" \
--tiny_imagenet_data_dir="${TINY_IMAGENET_DIR}"
```
Full list of training hyperparameters could be found in `model_lib.py`.
These hyperparameters control learning rate schedule, optimizer, weight decay,
label smoothing and adversarial training.
Adversarial training is controlled by following hyperparameters:
* `train_adv_method` - method which is used to craft adversarial examples during
training. Could be one of the following:
* `clean` - perform regular training with clean examples;
* `pgd_EPS_STEP_NITER` - use non targeted PGD with maximum size of
perturbation equal to `EPS`, step size equal to `STEP`
and number of iterations equal to `NITER`. Size of perturbation and step
size are expected to be integers between 1 and 255.
* `pgdll_EPS_STEP_NITER` - use targeted PGD, where target class is least
likely prediction of the network.
* `pgdrnd_EPS_STEP_NITER` - use targeted PGD, where target class is chosen
randomly.
* `train_lp_weight` - weight of adversarial logit pairing loss. If zero or
negarive, then no logit pairing is performed and training is done using
mixed minibatch PGD. If positive then adversarial logit pairing term is added
to the loss.
Below is example of how to run training with adversarial logit pairing on
ImageNet 64x64:
```
python train.py \
--model_name="resnet_v2_50" \
--hparams="train_adv_method=pgdll_16_2_10,train_lp_weight=0.5" \
--dataset="imagenet" \
--dataset_image_size=64 \
--output_dir="/tmp/adv_train" \
--imagenet_data_dir="${IMAGENET_DIR}"
```
### Fine tuning
Provided trainin script could be used to fine tune pre-trained checkpoint.
Following command does this:
```
# Fine tuning adds following additional arguments:
# - SCOPES_DO_NOT_LOAD_FROM_CHECKPOINT - comma separates list of scopes of
# variables, which should not be loadeded from checkpoint (and default
# initialization should be used instead).
# SCOPES_DO_NOT_LOAD_FROM_CHECKPOINT should be either same or a subset of
# LIST_OF_SCOPES_OF_TRAINABLE_VARS.
# - LIST_OF_SCOPES_OF_TRAINABLE_VARS - comma separated list of scopes of
# trainable variables. Only variables which are prefixed with these scopes
# will be trained.
# - PATH_TO_PRETRAINED_CHECKPOINT - directory with pretrained checkpoint which
# is used as initialization for fine tuning.
#
python train.py \
--max_steps="${MAX_NUMBER_OF_TRAINING_STEPS}" \
--model_name="${MODEL_NAME}" \
--moving_average_decay="${MOVING_AVG_DECAY}" \
--hparams="${HYPERPARAMETERS}" \
--dataset="${DATASET}" \
--dataset_image_size="${IMAGE_SIZE}" \
--output_dir="${OUTPUT_DIRECTORY}" \
--imagenet_data_dir="${IMAGENET_DIR}" \
--tiny_imagenet_data_dir="${TINY_IMAGENET_DIR}" \
--finetune_exclude_pretrained_scopes="${SCOPES_DO_NOT_LOAD_FROM_CHECKPOINT}" \
--finetune_trainable_scopes="${LIST_OF_SCOPES_OF_TRAINABLE_VARS}" \
--finetune_checkpoint_path="${PATH_TO_PRETRAINED_CHECKPOINT}"
```
Below is an example of how to fine tune last few layers of the model on
Tiny Imagenet dataset:
```
python train.py \
--model_name="resnet_v2_50" \
--hparams="train_adv_method=pgdll_16_2_10,train_lp_weight=0.5,learning_rate=0.02" \
--dataset="tiny_imagenet" \
--dataset_image_size=64 \
--output_dir="/tmp/adv_finetune" \
--tiny_imagenet_data_dir="${TINY_IMAGENET_DIR}" \
--finetune_exclude_pretrained_scopes="resnet_v2_50/logits" \
--finetune_trainable_scopes="resnet_v2_50/logits,resnet_v2_50/postnorm" \
--finetune_checkpoint_path="/tmp/adv_train"
```
### Evaluation
Following command runs evaluation:
```
# Following arguments should be provided for eval:
# - TRAINING_DIRECTORY - directory where training checkpoints are saved.
# - TRAINABLE_SCOPES - when loading checkpoint which was obtained by fine tuning
# this argument should be the same as LIST_OF_SCOPES_OF_TRAINABLE_VARS
# during training. Otherwise it should be empty.
# This is needed to properly load exponential moving average variables.
# If exponential moving averages are disabled then this flag could be
# omitted.
# - EVAL_SUBDIR_NAME - name of the subdirectory inside TRAINING_DIRECTORY
# where evaluation code will be saving event files.
# - DATASET - name of the dataset.
# - IMAGE_SIZE - size of the image in the dataset.
# - DATSET_SPLIT_NAME - name of the split in the dataset,
# either 'train' or 'validation'. Default is 'validation'.
# - MODEL_NAME - name of the model.
# - MOVING_AVG_DECAY - decay rate for exponential moving average.
# - ADV_METHOD_FOR_EVAL - should be "clean" to evaluate on clean example or
# description of the adversarial method to evaluate on adversarial examples.
# - HYPERPARAMETERS - hyperparameters, only "eval_batch_size" matters for eval
# - NUMBER_OF_EXAMPLES - how many examples from the dataset use for evaluation,
# specify -1 to use all examples.
# - EVAL_ONCE - if True then evaluate only once, otherwise keep evaluation
# running repeatedly on new checkpoints. Repeated evaluation might be useful
# when running concurrent with training.
# - IMAGENET_DIR - directory with ImageNet dataset in TFRecord format.
# - TINY_IMAGENET_DIR - directory with Tiny ImageNet dataset in TFRecord format.
#
python eval.py \
--train_dir="${TRAINING_DIRECTORY} \
--trainable_scopes="${TRAINABLE_SCOPES}" \
--eval_name="${EVAL_SUBDIR_NAME}" \
--dataset="${DATASET}" \
--dataset_image_size="${IMAGE_SIZE}" \
--split_name="${DATSET_SPLIT_NAME}" \
--model_name="${MODEL_NAME}" \
--moving_average_decay="${MOVING_AVG_DECAY}" \
--adv_method="${ADV_METHOD_FOR_EVAL}" \
--hparams="${HYPERPARAMETERS}" \
--num_examples="${NUMBER_OF_EXAMPLES}" \
--eval_once="${EVAL_ONCE}" \
--imagenet_data_dir="${IMAGENET_DIR}" \
--tiny_imagenet_data_dir="${TINY_IMAGENET_DIR}"
```
Example of running evaluation on 10000 of clean examples from ImageNet
training set:
```
python eval.py \
--train_dir=/tmp/adv_train \
--dataset=imagenet \
--dataset_image_size=64 \
--split_name=train \
--adv_method=clean \
--hparams="eval_batch_size=50" \
--num_examples=10000 \
--eval_once=True \
--imagenet_data_dir="${IMAGENET_DIR}"
```
Example of running evaluatin on adversarial images generated from Tiny ImageNet
validation set using fine-tuned checkpoint:
```
python eval.py \
--train_dir=tmp/adv_finetune \
--trainable_scopes="resnet_v2_50/logits,resnet_v2_50/postnorm" \
--dataset=tiny_imagenet \
--dataset_image_size=64 \
--adv_method=pgdrnd_16_2_10 \
--hparams="eval_batch_size=50" \
--eval_once=True \
--tiny_imagenet_data_dir="${TINY_IMAGENET_DIR}"
```
### Pre-trained models
Following set of pre-trained checkpoints released with this code:
| Model | Dataset | Accuracy on<br>clean images | Accuracy on<br>`pgdll_16_1_20` | Accuracy on<br>`pgdll_16_2_10` |
| ----------- | ------------ | --------------- | --------------------------- | -------------- |
| [Baseline ResNet-v2-50](http://download.tensorflow.org/models/adversarial_logit_pairing/imagenet64_base_2018_06_26.ckpt.tar.gz) | ImageNet 64x64 | 60.5% | 1.8% | 3.5% |
| [ALP-trained ResNet-v2-50](http://download.tensorflow.org/models/adversarial_logit_pairing/imagenet64_alp025_2018_06_26.ckpt.tar.gz) | ImageNet 64x64 | 55.7% | 27.5% | 27.8% |
| [Baseline ResNet-v2-50](http://download.tensorflow.org/models/adversarial_logit_pairing/tiny_imagenet_base_2018_06_26.ckpt.tar.gz) | Tiny ImageNet | 69.2% | 0.1% | 0.3% |
| [ALP-trained ResNet-v2-50](http://download.tensorflow.org/models/adversarial_logit_pairing/tiny_imagenet_alp05_2018_06_26.ckpt.tar.gz) | Tiny ImageNet | 72.0% | 41.3% | 40.8% |
* All provided checkpoints were initially trained with exponential moving
average. However for ease of use they were re-saved without it.
So to load and use provided checkpoints you need to specify
`--moving_average_decay=-1` flag.
* All ALP models were trained with `pgdll_16_2_10` adversarial examples.
* All Tiny Imagenet models were obtained by fine tuning corresponding
ImageNet 64x64 models. ALP-trained models were fine tuned with ALP.
research/adversarial_logit_pairing/adversarial_attack.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2018 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Library with adversarial attacks.
This library designed to be self-contained and have no dependencies other
than TensorFlow. It only contains PGD / Iterative FGSM attacks,
see https://arxiv.org/abs/1706.06083 and https://arxiv.org/abs/1607.02533
for details.
For wider set of adversarial attacks refer to Cleverhans library:
https://github.com/tensorflow/cleverhans
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import tensorflow as tf
def generate_pgd_common(x,
bounds,
model_fn,
attack_params,
one_hot_labels,
perturbation_multiplier):
"""Common code for generating PGD adversarial examples.
Args:
x: original examples.
bounds: tuple with bounds of image values, bounds[0] < bounds[1].
model_fn: model function with signature model_fn(images).
attack_params: parameters of the attack.
one_hot_labels: one hot label vector to use in the loss.
perturbation_multiplier: multiplier of adversarial perturbation,
either +1.0 or -1.0.
Returns:
Tensor with adversarial examples.
Raises:
ValueError: if attack parameters are invalid.
"""
# parse attack_params
# Format of attack_params: 'EPS_STEP_NITER'
# where EPS - epsilon, STEP - step size, NITER - number of iterations
params_list = attack_params.split('_')
if len(params_list) != 3:
raise ValueError('Invalid parameters of PGD attack: %s' % attack_params)
epsilon = int(params_list[0])
step_size = int(params_list[1])
niter = int(params_list[2])
# rescale epsilon and step size to image bounds
epsilon = float(epsilon) / 255.0 * (bounds[1] - bounds[0])
step_size = float(step_size) / 255.0 * (bounds[1] - bounds[0])
# clipping boundaries
clip_min = tf.maximum(x - epsilon, bounds[0])
clip_max = tf.minimum(x + epsilon, bounds[1])
# compute starting point
start_x = x + tf.random_uniform(tf.shape(x), -epsilon, epsilon)
start_x = tf.clip_by_value(start_x, clip_min, clip_max)
# main iteration of PGD
loop_vars = [0, start_x]
def loop_cond(index, _):
return index < niter
def loop_body(index, adv_images):
logits = model_fn(adv_images)
loss = tf.reduce_sum(
tf.nn.softmax_cross_entropy_with_logits_v2(
labels=one_hot_labels,
logits=logits))
perturbation = step_size * tf.sign(tf.gradients(loss, adv_images)[0])
new_adv_images = adv_images + perturbation_multiplier * perturbation
new_adv_images = tf.clip_by_value(new_adv_images, clip_min, clip_max)
return index + 1, new_adv_images
with tf.control_dependencies([start_x]):
_, result = tf.while_loop(
loop_cond,
loop_body,
loop_vars,
back_prop=False,
parallel_iterations=1)
return result
def generate_pgd_ll(x, bounds, model_fn, attack_params):
# pylint: disable=g-doc-args
"""Generats targeted PGD adversarial examples with least likely target class.
See generate_pgd_common for description of arguments.
Returns:
Tensor with adversarial examples.
"""
# pylint: enable=g-doc-args
# compute one hot least likely class
logits = model_fn(x)
num_classes = tf.shape(logits)[1]
one_hot_labels = tf.one_hot(tf.argmin(model_fn(x), axis=1), num_classes)
return generate_pgd_common(x, bounds, model_fn, attack_params,
one_hot_labels=one_hot_labels,
perturbation_multiplier=-1.0)
def generate_pgd_rand(x, bounds, model_fn, attack_params):
# pylint: disable=g-doc-args
"""Generats targeted PGD adversarial examples with random target class.
See generate_pgd_common for description of arguments.
Returns:
Tensor with adversarial examples.
"""
# pylint: enable=g-doc-args
# compute one hot random class
logits = model_fn(x)
batch_size = tf.shape(logits)[0]
num_classes = tf.shape(logits)[1]
random_labels = tf.random_uniform(shape=[batch_size],
minval=0,
maxval=num_classes,
dtype=tf.int32)
one_hot_labels = tf.one_hot(random_labels, num_classes)
return generate_pgd_common(x, bounds, model_fn, attack_params,
one_hot_labels=one_hot_labels,
perturbation_multiplier=-1.0)
def generate_pgd(x, bounds, model_fn, attack_params):
# pylint: disable=g-doc-args
"""Generats non-targeted PGD adversarial examples.
See generate_pgd_common for description of arguments.
Returns:
tensor with adversarial examples.
"""
# pylint: enable=g-doc-args
# compute one hot predicted class
logits = model_fn(x)
num_classes = tf.shape(logits)[1]
one_hot_labels = tf.one_hot(tf.argmax(model_fn(x), axis=1), num_classes)
return generate_pgd_common(x, bounds, model_fn, attack_params,
one_hot_labels=one_hot_labels,
perturbation_multiplier=1.0)
def generate_adversarial_examples(x, bounds, model_fn, attack_description):
"""Generates adversarial examples.
Args:
x: original examples.
bounds: tuple with bounds of image values, bounds[0] < bounds[1]
model_fn: model function with signature model_fn(images).
attack_description: string which describes an attack, see notes below for
details.
Returns:
Tensor with adversarial examples.
Raises:
ValueError: if attack description is invalid.
Attack description could be one of the following strings:
- "clean" - no attack, return original images.
- "pgd_EPS_STEP_NITER" - non-targeted PGD attack.
- "pgdll_EPS_STEP_NITER" - tageted PGD attack with least likely target class.
- "pgdrnd_EPS_STEP_NITER" - targetd PGD attack with random target class.
Meaning of attack parameters is following:
- EPS - maximum size of adversarial perturbation, between 0 and 255.
- STEP - step size of one iteration of PGD, between 0 and 255.
- NITER - number of iterations.
"""
if attack_description == 'clean':
return x
idx = attack_description.find('_')
if idx < 0:
raise ValueError('Invalid value of attack description %s'
% attack_description)
attack_name = attack_description[:idx]
attack_params = attack_description[idx+1:]
if attack_name == 'pgdll':
return generate_pgd_ll(x, bounds, model_fn, attack_params)
elif attack_name == 'pgdrnd':
return generate_pgd_rand(x, bounds, model_fn, attack_params)
elif attack_name == 'pgd':
return generate_pgd(x, bounds, model_fn, attack_params)
else:
raise ValueError('Invalid value of attack description %s'
% attack_description)
research/adversarial_logit_pairing/datasets/dataset_factory.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2018 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Library which creates datasets."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from datasets import imagenet_input
from datasets import tiny_imagenet_input
def get_dataset(dataset_name, split, batch_size, image_size, is_training):
"""Returns dataset.
Args:
dataset_name: name of the dataset, "imagenet" or "tiny_imagenet".
split: name of the split, "train" or "validation".
batch_size: size of the minibatch.
image_size: size of the one side of the image. Output images will be
resized to square shape image_size*image_size.
is_training: if True then training preprocessing is done, otherwise eval
preprocessing is done.
Raises:
ValueError: if dataset_name is invalid.
Returns:
dataset: instance of tf.data.Dataset with the dataset.
num_examples: number of examples in given split of the dataset.
num_classes: number of classes in the dataset.
bounds: tuple with bounds of image values. All returned image pixels
are between bounds[0] and bounds[1].
"""
if dataset_name == 'tiny_imagenet':
dataset = tiny_imagenet_input.tiny_imagenet_input(
split, batch_size, image_size, is_training)
num_examples = tiny_imagenet_input.num_examples_per_epoch(split)
num_classes = 200
bounds = (-1, 1)
elif dataset_name == 'imagenet':
dataset = imagenet_input.imagenet_input(
split, batch_size, image_size, is_training)
num_examples = imagenet_input.num_examples_per_epoch(split)
num_classes = 1001
bounds = (-1, 1)
else:
raise ValueError('Invalid dataset %s' % dataset_name)
return dataset, num_examples, num_classes, bounds
research/adversarial_logit_pairing/datasets/imagenet_input.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2018 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Imagenet input."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
from absl import flags
import tensorflow as tf
FLAGS = flags.FLAGS
flags.DEFINE_string('imagenet_data_dir', None,
'Directory with Imagenet dataset in TFRecord format.')
def _decode_and_random_crop(image_buffer, bbox, image_size):
"""Randomly crops image and then scales to target size."""
with tf.name_scope('distorted_bounding_box_crop',
values=[image_buffer, bbox]):
sample_distorted_bounding_box = tf.image.sample_distorted_bounding_box(
tf.image.extract_jpeg_shape(image_buffer),
bounding_boxes=bbox,
min_object_covered=0.1,
aspect_ratio_range=[0.75, 1.33],
area_range=[0.08, 1.0],
max_attempts=10,
use_image_if_no_bounding_boxes=True)
bbox_begin, bbox_size, _ = sample_distorted_bounding_box
# Crop the image to the specified bounding box.
offset_y, offset_x, _ = tf.unstack(bbox_begin)
target_height, target_width, _ = tf.unstack(bbox_size)
crop_window = tf.stack([offset_y, offset_x, target_height, target_width])
image = tf.image.decode_and_crop_jpeg(image_buffer, crop_window, channels=3)
image = tf.image.convert_image_dtype(
image, dtype=tf.float32)
image = tf.image.resize_bicubic([image],
[image_size, image_size])[0]
return image
def _decode_and_center_crop(image_buffer, image_size):
"""Crops to center of image with padding then scales to target size."""
shape = tf.image.extract_jpeg_shape(image_buffer)
image_height = shape[0]
image_width = shape[1]
padded_center_crop_size = tf.cast(
0.875 * tf.cast(tf.minimum(image_height, image_width), tf.float32),
tf.int32)
offset_height = ((image_height - padded_center_crop_size) + 1) // 2
offset_width = ((image_width - padded_center_crop_size) + 1) // 2
crop_window = tf.stack([offset_height, offset_width,
padded_center_crop_size, padded_center_crop_size])
image = tf.image.decode_and_crop_jpeg(image_buffer, crop_window, channels=3)
image = tf.image.convert_image_dtype(
image, dtype=tf.float32)
image = tf.image.resize_bicubic([image],
[image_size, image_size])[0]
return image
def _normalize(image):
"""Rescale image to [-1, 1] range."""
return tf.multiply(tf.subtract(image, 0.5), 2.0)
def image_preprocessing(image_buffer, bbox, image_size, is_training):
"""Does image decoding and preprocessing.
Args:
image_buffer: string tensor with encoded image.
bbox: bounding box of the object at the image.
image_size: image size.
is_training: whether to do training or eval preprocessing.
Returns:
Tensor with the image.
"""
if is_training:
image = _decode_and_random_crop(image_buffer, bbox, image_size)
image = _normalize(image)
image = tf.image.random_flip_left_right(image)
else:
image = _decode_and_center_crop(image_buffer, image_size)
image = _normalize(image)
image = tf.reshape(image, [image_size, image_size, 3])
return image
def imagenet_parser(value, image_size, is_training):
"""Parse an ImageNet record from a serialized string Tensor.
Args:
value: encoded example.
image_size: size of the output image.
is_training: if True then do training preprocessing,
otherwise do eval preprocessing.
Returns:
image: tensor with the image.
label: true label of the image.
"""
keys_to_features = {
'image/encoded':
tf.FixedLenFeature((), tf.string, ''),
'image/format':
tf.FixedLenFeature((), tf.string, 'jpeg'),
'image/class/label':
tf.FixedLenFeature([], tf.int64, -1),
'image/class/text':
tf.FixedLenFeature([], tf.string, ''),
'image/object/bbox/xmin':
tf.VarLenFeature(dtype=tf.float32),
'image/object/bbox/ymin':
tf.VarLenFeature(dtype=tf.float32),
'image/object/bbox/xmax':
tf.VarLenFeature(dtype=tf.float32),
'image/object/bbox/ymax':
tf.VarLenFeature(dtype=tf.float32),
'image/object/class/label':
tf.VarLenFeature(dtype=tf.int64),
}
parsed = tf.parse_single_example(value, keys_to_features)
image_buffer = tf.reshape(parsed['image/encoded'], shape=[])
xmin = tf.expand_dims(parsed['image/object/bbox/xmin'].values, 0)
ymin = tf.expand_dims(parsed['image/object/bbox/ymin'].values, 0)
xmax = tf.expand_dims(parsed['image/object/bbox/xmax'].values, 0)
ymax = tf.expand_dims(parsed['image/object/bbox/ymax'].values, 0)
# Note that ordering is (y, x)
bbox = tf.concat([ymin, xmin, ymax, xmax], 0)
# Force the variable number of bounding boxes into the shape
# [1, num_boxes, coords].
bbox = tf.expand_dims(bbox, 0)
bbox = tf.transpose(bbox, [0, 2, 1])
image = image_preprocessing(
image_buffer=image_buffer,
bbox=bbox,
image_size=image_size,
is_training=is_training
)
# Labels are in [1, 1000] range
label = tf.cast(
tf.reshape(parsed['image/class/label'], shape=[]), dtype=tf.int32)
return image, label
def imagenet_input(split, batch_size, image_size, is_training):
"""Returns ImageNet dataset.
Args:
split: name of the split, "train" or "validation".
batch_size: size of the minibatch.
image_size: size of the one side of the image. Output images will be
resized to square shape image_size*image_size.
is_training: if True then training preprocessing is done, otherwise eval
preprocessing is done.
Raises:
ValueError: if name of the split is incorrect.
Returns:
Instance of tf.data.Dataset with the dataset.
"""
if split.lower().startswith('train'):
file_pattern = os.path.join(FLAGS.imagenet_data_dir, 'train-*')
elif split.lower().startswith('validation'):
file_pattern = os.path.join(FLAGS.imagenet_data_dir, 'validation-*')
else:
raise ValueError('Invalid split: %s' % split)
dataset = tf.data.Dataset.list_files(file_pattern, shuffle=is_training)
if is_training:
dataset = dataset.repeat()
def fetch_dataset(filename):
return tf.data.TFRecordDataset(filename, buffer_size=8*1024*1024)
# Read the data from disk in parallel
dataset = dataset.apply(
tf.data.experimental.parallel_interleave(
fetch_dataset, cycle_length=4, sloppy=True))
dataset = dataset.shuffle(1024)
# Parse, preprocess, and batch the data in parallel
dataset = dataset.apply(
tf.data.experimental.map_and_batch(
lambda value: imagenet_parser(value, image_size, is_training),
batch_size=batch_size,
num_parallel_batches=4,
drop_remainder=True))
def set_shapes(images, labels):
"""Statically set the batch_size dimension."""
images.set_shape(images.get_shape().merge_with(
tf.TensorShape([batch_size, None, None, None])))
labels.set_shape(labels.get_shape().merge_with(
tf.TensorShape([batch_size])))
return images, labels
# Assign static batch size dimension
dataset = dataset.map(set_shapes)
# Prefetch overlaps in-feed with training
dataset = dataset.prefetch(tf.data.experimental.AUTOTUNE)
return dataset
def num_examples_per_epoch(split):
"""Returns the number of examples in the data set.
Args:
split: name of the split, "train" or "validation".
Raises:
ValueError: if split name is incorrect.
Returns:
Number of example in the split.
"""
if split.lower().startswith('train'):
return 1281167
elif split.lower().startswith('validation'):
return 50000
else:
raise ValueError('Invalid split: %s' % split)
research/adversarial_logit_pairing/datasets/tiny_imagenet_input.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2018 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Tiny imagenet input."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
from absl import flags
import tensorflow as tf
FLAGS = flags.FLAGS
flags.DEFINE_string('tiny_imagenet_data_dir', None,
'Directory with Tiny Imagenet dataset in TFRecord format.')
def tiny_imagenet_parser(value, image_size, is_training):
"""Parses tiny imagenet example.
Args:
value: encoded example.
image_size: size of the image.
is_training: if True then do training preprocessing (which includes
random cropping), otherwise do eval preprocessing.
Returns:
image: tensor with the image.
label: true label of the image.
"""
keys_to_features = {
'image/encoded': tf.FixedLenFeature((), tf.string, ''),
'label/tiny_imagenet': tf.FixedLenFeature([], tf.int64, -1),
}
parsed = tf.parse_single_example(value, keys_to_features)
image_buffer = tf.reshape(parsed['image/encoded'], shape=[])
image = tf.image.decode_image(image_buffer, channels=3)
image = tf.image.convert_image_dtype(
image, dtype=tf.float32)
# Crop image
if is_training:
bbox_begin, bbox_size, _ = tf.image.sample_distorted_bounding_box(
tf.shape(image),
bounding_boxes=tf.constant([0.0, 0.0, 1.0, 1.0],
dtype=tf.float32,
shape=[1, 1, 4]),
min_object_covered=0.5,
aspect_ratio_range=[0.75, 1.33],
area_range=[0.5, 1.0],
max_attempts=20,
use_image_if_no_bounding_boxes=True)
image = tf.slice(image, bbox_begin, bbox_size)
# resize image
image = tf.image.resize_bicubic([image], [image_size, image_size])[0]
# Rescale image to [-1, 1] range.
image = tf.multiply(tf.subtract(image, 0.5), 2.0)
image = tf.reshape(image, [image_size, image_size, 3])
# Labels are in [0, 199] range
label = tf.cast(
tf.reshape(parsed['label/tiny_imagenet'], shape=[]), dtype=tf.int32)
return image, label
def tiny_imagenet_input(split, batch_size, image_size, is_training):
"""Returns Tiny Imagenet Dataset.
Args:
split: name of the split, "train" or "validation".
batch_size: size of the minibatch.
image_size: size of the one side of the image. Output images will be
resized to square shape image_size*image_size.
is_training: if True then training preprocessing is done, otherwise eval
preprocessing is done.instance of tf.data.Dataset with the dataset.
Raises:
ValueError: if name of the split is incorrect.
Returns:
Instance of tf.data.Dataset with the dataset.
"""
if split.lower().startswith('train'):
filepath = os.path.join(FLAGS.tiny_imagenet_data_dir, 'train.tfrecord')
elif split.lower().startswith('validation'):
filepath = os.path.join(FLAGS.tiny_imagenet_data_dir, 'validation.tfrecord')
else:
raise ValueError('Invalid split: %s' % split)
dataset = tf.data.TFRecordDataset(filepath, buffer_size=8*1024*1024)
if is_training:
dataset = dataset.shuffle(10000)
dataset = dataset.repeat()
dataset = dataset.apply(
tf.data.experimental.map_and_batch(
lambda value: tiny_imagenet_parser(value, image_size, is_training),
batch_size=batch_size,
num_parallel_batches=4,
drop_remainder=True))
def set_shapes(images, labels):
"""Statically set the batch_size dimension."""
images.set_shape(images.get_shape().merge_with(
tf.TensorShape([batch_size, None, None, None])))
labels.set_shape(labels.get_shape().merge_with(
tf.TensorShape([batch_size])))
return images, labels
# Assign static batch size dimension
dataset = dataset.map(set_shapes)
dataset = dataset.prefetch(tf.data.experimental.AUTOTUNE)
return dataset
def num_examples_per_epoch(split):
"""Returns the number of examples in the data set.
Args:
split: name of the split, "train" or "validation".
Raises:
ValueError: if split name is incorrect.
Returns:
Number of example in the split.
"""
if split.lower().startswith('train'):
return 100000
elif split.lower().startswith('validation'):
return 10000
else:
raise ValueError('Invalid split: %s' % split)
research/adversarial_logit_pairing/eval.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2018 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Program which runs evaluation of Imagenet 64x64 and TinyImagenet models."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
from absl import app
from absl import flags
import tensorflow as tf
import adversarial_attack
import model_lib
from datasets import dataset_factory
FLAGS = flags.FLAGS
flags.DEFINE_string('train_dir', None,
'Training directory. If specified then this program '
'runs in continuous evaluation mode.')
flags.DEFINE_string('checkpoint_path', None,
'Path to the file with checkpoint. If specified then '
'this program evaluates only provided checkpoint one time.')
flags.DEFINE_string('output_file', None,
'Name of output file. Used only in single evaluation mode.')
flags.DEFINE_string('eval_name', 'default', 'Name for eval subdirectory.')
flags.DEFINE_string('master', '', 'Tensorflow master.')
flags.DEFINE_string('model_name', 'resnet_v2_50', 'Name of the model.')
flags.DEFINE_string('adv_method', 'clean',
'Method which is used to generate adversarial examples.')
flags.DEFINE_string('dataset', 'imagenet',
'Dataset: "tiny_imagenet" or "imagenet".')
flags.DEFINE_integer('dataset_image_size', 64,
'Size of the images in the dataset.')
flags.DEFINE_string('hparams', '', 'Hyper parameters.')
flags.DEFINE_string('split_name', 'validation', 'Name of the split.')
flags.DEFINE_float('moving_average_decay', 0.9999,
'The decay to use for the moving average.')
flags.DEFINE_integer('eval_interval_secs', 120,
'The frequency, in seconds, with which evaluation is run.')
flags.DEFINE_integer(
'num_examples', -1,
'If positive - maximum number of example to use for evaluation.')
flags.DEFINE_bool('eval_once', False,
'If true then evaluate model only once.')
flags.DEFINE_string('trainable_scopes', None,
'If set then it defines list of variable scopes for '
'trainable variables.')
def main(_):
if not FLAGS.train_dir and not FLAGS.checkpoint_path:
print('Either --train_dir or --checkpoint_path flags has to be provided.')
if FLAGS.train_dir and FLAGS.checkpoint_path:
print('Only one of --train_dir or --checkpoint_path should be provided.')
params = model_lib.default_hparams()
params.parse(FLAGS.hparams)
tf.logging.info('User provided hparams: %s', FLAGS.hparams)
tf.logging.info('All hyper parameters: %s', params)
batch_size = params.eval_batch_size
graph = tf.Graph()
with graph.as_default():
# dataset
dataset, num_examples, num_classes, bounds = dataset_factory.get_dataset(
FLAGS.dataset,
FLAGS.split_name,
batch_size,
FLAGS.dataset_image_size,
is_training=False)
dataset_iterator = dataset.make_one_shot_iterator()
images, labels = dataset_iterator.get_next()
if FLAGS.num_examples > 0:
num_examples = min(num_examples, FLAGS.num_examples)
# setup model
global_step = tf.train.get_or_create_global_step()
model_fn_two_args = model_lib.get_model(FLAGS.model_name, num_classes)
model_fn = lambda x: model_fn_two_args(x, is_training=False)
if not FLAGS.adv_method or FLAGS.adv_method == 'clean':
logits = model_fn(images)
else:
adv_examples = adversarial_attack.generate_adversarial_examples(
images, bounds, model_fn, FLAGS.adv_method)
logits = model_fn(adv_examples)
# update trainable variables if fine tuning is used
model_lib.filter_trainable_variables(FLAGS.trainable_scopes)
# Setup the moving averages
if FLAGS.moving_average_decay and (FLAGS.moving_average_decay > 0):
variable_averages = tf.train.ExponentialMovingAverage(
FLAGS.moving_average_decay, global_step)
variables_to_restore = variable_averages.variables_to_restore(
tf.contrib.framework.get_model_variables())
variables_to_restore[global_step.op.name] = global_step
else:
variables_to_restore = tf.contrib.framework.get_variables_to_restore()
# Setup evaluation metric
with tf.name_scope('Eval'):
names_to_values, names_to_updates = (
tf.contrib.metrics.aggregate_metric_map({
'Accuracy': tf.metrics.accuracy(labels, tf.argmax(logits, 1)),
'Top5': tf.metrics.recall_at_k(tf.to_int64(labels), logits, 5)
}))
for name, value in names_to_values.iteritems():
tf.summary.scalar(name, value)
# Run evaluation
num_batches = int(num_examples / batch_size)
if FLAGS.train_dir:
output_dir = os.path.join(FLAGS.train_dir, FLAGS.eval_name)
if not tf.gfile.Exists(output_dir):
tf.gfile.MakeDirs(output_dir)
tf.contrib.training.evaluate_repeatedly(
FLAGS.train_dir,
master=FLAGS.master,
scaffold=tf.train.Scaffold(
saver=tf.train.Saver(variables_to_restore)),
eval_ops=names_to_updates.values(),
eval_interval_secs=FLAGS.eval_interval_secs,
hooks=[
tf.contrib.training.StopAfterNEvalsHook(num_batches),
tf.contrib.training.SummaryAtEndHook(output_dir),
tf.train.LoggingTensorHook(names_to_values, at_end=True),
],
max_number_of_evaluations=1 if FLAGS.eval_once else None)
else:
result = tf.contrib.training.evaluate_once(
FLAGS.checkpoint_path,
master=FLAGS.master,
scaffold=tf.train.Scaffold(
saver=tf.train.Saver(variables_to_restore)),
eval_ops=names_to_updates.values(),
final_ops=names_to_values,
hooks=[
tf.contrib.training.StopAfterNEvalsHook(num_batches),
tf.train.LoggingTensorHook(names_to_values, at_end=True),
])
if FLAGS.output_file:
with tf.gfile.Open(FLAGS.output_file, 'a') as f:
f.write('%s,%.3f,%.3f\n'
% (FLAGS.eval_name, result['Accuracy'], result['Top5']))
if __name__ == '__main__':
app.run(main)
research/adversarial_logit_pairing/model_lib.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2018 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Library with common functions for training and eval."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import six
import tensorflow as tf
from tensorflow.contrib.slim.nets import resnet_v2
def default_hparams():
"""Returns default hyperparameters."""
return tf.contrib.training.HParams(
# Batch size for training and evaluation.
batch_size=32,
eval_batch_size=50,
# General training parameters.
weight_decay=0.0001,
label_smoothing=0.1,
# Parameters of the adversarial training.
train_adv_method='clean', # adversarial training method
train_lp_weight=0.0, # Weight of adversarial logit pairing loss
# Parameters of the optimizer.
optimizer='rms', # possible values are: 'rms', 'momentum', 'adam'
momentum=0.9, # momentum
rmsprop_decay=0.9, # Decay term for RMSProp
rmsprop_epsilon=1.0, # Epsilon term for RMSProp
# Parameters of learning rate schedule.
lr_schedule='exp_decay', # Possible values: 'exp_decay', 'step', 'fixed'
learning_rate=0.045,
lr_decay_factor=0.94, # Learning exponential decay
lr_num_epochs_per_decay=2.0, # Number of epochs per lr decay
lr_list=[1.0 / 6, 2.0 / 6, 3.0 / 6,
4.0 / 6, 5.0 / 6, 1.0, 0.1, 0.01,
0.001, 0.0001],
lr_decay_epochs=[1, 2, 3, 4, 5, 30, 60, 80,
90])
def get_lr_schedule(hparams, examples_per_epoch, replicas_to_aggregate=1):
"""Returns TensorFlow op which compute learning rate.
Args:
hparams: hyper parameters.
examples_per_epoch: number of training examples per epoch.
replicas_to_aggregate: number of training replicas running in parallel.
Raises:
ValueError: if learning rate schedule specified in hparams is incorrect.
Returns:
learning_rate: tensor with learning rate.
steps_per_epoch: number of training steps per epoch.
"""
global_step = tf.train.get_or_create_global_step()
steps_per_epoch = float(examples_per_epoch) / float(hparams.batch_size)
if replicas_to_aggregate > 0:
steps_per_epoch /= replicas_to_aggregate
if hparams.lr_schedule == 'exp_decay':
decay_steps = long(steps_per_epoch * hparams.lr_num_epochs_per_decay)
learning_rate = tf.train.exponential_decay(
hparams.learning_rate,
global_step,
decay_steps,
hparams.lr_decay_factor,
staircase=True)
elif hparams.lr_schedule == 'step':
lr_decay_steps = [long(epoch * steps_per_epoch)
for epoch in hparams.lr_decay_epochs]
learning_rate = tf.train.piecewise_constant(
global_step, lr_decay_steps, hparams.lr_list)
elif hparams.lr_schedule == 'fixed':
learning_rate = hparams.learning_rate
else:
raise ValueError('Invalid value of lr_schedule: %s' % hparams.lr_schedule)
if replicas_to_aggregate > 0:
learning_rate *= replicas_to_aggregate
return learning_rate, steps_per_epoch
def get_optimizer(hparams, learning_rate):
"""Returns optimizer.
Args:
hparams: hyper parameters.
learning_rate: learning rate tensor.
Raises:
ValueError: if type of optimizer specified in hparams is incorrect.
Returns:
Instance of optimizer class.
"""
if hparams.optimizer == 'rms':
optimizer = tf.train.RMSPropOptimizer(learning_rate,
hparams.rmsprop_decay,
hparams.momentum,
hparams.rmsprop_epsilon)
elif hparams.optimizer == 'momentum':
optimizer = tf.train.MomentumOptimizer(learning_rate,
hparams.momentum)
elif hparams.optimizer == 'adam':
optimizer = tf.train.AdamOptimizer(learning_rate)
else:
raise ValueError('Invalid value of optimizer: %s' % hparams.optimizer)
return optimizer
RESNET_MODELS = {'resnet_v2_50': resnet_v2.resnet_v2_50}
def get_model(model_name, num_classes):
"""Returns function which creates model.
Args:
model_name: Name of the model.
num_classes: Number of classes.
Raises:
ValueError: If model_name is invalid.
Returns:
Function, which creates model when called.
"""
if model_name.startswith('resnet'):
def resnet_model(images, is_training, reuse=tf.AUTO_REUSE):
with tf.contrib.framework.arg_scope(resnet_v2.resnet_arg_scope()):
resnet_fn = RESNET_MODELS[model_name]
logits, _ = resnet_fn(images, num_classes, is_training=is_training,
reuse=reuse)
logits = tf.reshape(logits, [-1, num_classes])
return logits
return resnet_model
else:
raise ValueError('Invalid model: %s' % model_name)
def filter_trainable_variables(trainable_scopes):
"""Keep only trainable variables which are prefixed with given scopes.
Args:
trainable_scopes: either list of trainable scopes or string with comma
separated list of trainable scopes.
This function removes all variables which are not prefixed with given
trainable_scopes from collection of trainable variables.
Useful during network fine tuning, when you only need to train subset of
variables.
"""
if not trainable_scopes:
return
if isinstance(trainable_scopes, six.string_types):
trainable_scopes = [scope.strip() for scope in trainable_scopes.split(',')]
trainable_scopes = {scope for scope in trainable_scopes if scope}
if not trainable_scopes:
return
trainable_collection = tf.get_collection_ref(
tf.GraphKeys.TRAINABLE_VARIABLES)
non_trainable_vars = [
v for v in trainable_collection
if not any([v.op.name.startswith(s) for s in trainable_scopes])
]
for v in non_trainable_vars:
trainable_collection.remove(v)
research/adversarial_logit_pairing/tiny_imagenet_converter/converter.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2018 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Converts Tiny Imagenet dataset into TFRecord format.
As an output this program generates following files in TFRecord format:
- train.tfrecord
- validation.tfrecord
- test.tfrecord
Generated train and validation files will contain tf.Example entries with
following features:
- image/encoded - encoded image
- image/format - image format
- label/wnid - label WordNet ID
- label/imagenet - imagenet label [1 ... 1000]
- label/tiny_imagenet - tiny imagenet label [0 ... 199]
- bbox/xmin
- bbox/ymin
- bbox/xmax
- bbox/ymax
Test file will contain entries with 'image/encoded' and 'image/format' features.
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from collections import namedtuple
import os
import random
from absl import app
from absl import flags
from absl import logging
import pandas as pd
import tensorflow as tf
FLAGS = flags.FLAGS
flags.DEFINE_string('input_dir', '', 'Input directory')
flags.DEFINE_string('output_dir', '', 'Output directory')
flags.DEFINE_string('imagenet_synsets_path', '',
'Optional path to /imagenet_lsvrc_2015_synsets.txt')
ImageMetadata = namedtuple('ImageMetadata', ['label', 'x1', 'y1', 'x2', 'y2'])
class WnIdToNodeIdConverter(object):
"""Converts WordNet IDs to numerical labels."""
def __init__(self, wnids_path, background_class):
self._wnid_to_node_id = {}
self._node_id_to_wnid = {}
with tf.gfile.Open(wnids_path) as f:
wnids_sequence = [wnid.strip() for wnid in f.readlines() if wnid.strip()]
node_id_offset = 1 if background_class else 0
for i, label in enumerate(wnids_sequence):
self._wnid_to_node_id[label] = i + node_id_offset
self._node_id_to_wnid[i + node_id_offset] = label
def to_node_id(self, wnid):
return self._wnid_to_node_id[wnid]
def to_wnid(self, node_id):
return self._node_id_to_wnid[node_id]
def all_wnids(self):
return self._wnid_to_node_id.keys()
def read_tiny_imagenet_annotations(annotations_filename,
images_dir,
one_label=None):
"""Reads one file with Tiny Imagenet annotations."""
result = []
if one_label:
column_names = ['filename', 'x1', 'y1', 'x2', 'y2']
else:
column_names = ['filename', 'label', 'x1', 'y1', 'x2', 'y2']
with tf.gfile.Open(annotations_filename) as f:
data = pd.read_csv(f, sep='\t', names=column_names)
for row in data.itertuples():
label = one_label if one_label else getattr(row, 'label')
full_filename = os.path.join(images_dir, getattr(row, 'filename'))
result.append((full_filename,
ImageMetadata(label=label,
x1=getattr(row, 'x1'),
y1=getattr(row, 'y1'),
x2=getattr(row, 'x2'),
y2=getattr(row, 'y2'))))
return result
def read_validation_annotations(validation_dir):
"""Reads validation data annotations."""
return read_tiny_imagenet_annotations(
os.path.join(validation_dir, 'val_annotations.txt'),
os.path.join(validation_dir, 'images'))
def read_training_annotations(training_dir):
"""Reads training data annotations."""
result = []
sub_dirs = tf.gfile.ListDirectory(training_dir)
for sub_dir in sub_dirs:
if not sub_dir.startswith('n'):
logging.warning('Found non-class directory in training dir: %s', sub_dir)
continue
sub_dir_results = read_tiny_imagenet_annotations(
os.path.join(training_dir, sub_dir, sub_dir + '_boxes.txt'),
os.path.join(training_dir, sub_dir, 'images'),
one_label=sub_dir)
result.extend(sub_dir_results)
return result
def read_test_annotations(test_dir):
"""Reads test data annotations."""
files = tf.gfile.ListDirectory(os.path.join(test_dir, 'images'))
return [(os.path.join(test_dir, 'images', f), None)
for f in files if f.endswith('.JPEG')]
def get_image_format(filename):
"""Returns image format from filename."""
filename = filename.lower()
if filename.endswith('jpeg') or filename.endswith('jpg'):
return 'jpeg'
elif filename.endswith('png'):
return 'png'
else:
raise ValueError('Unrecognized file format: %s' % filename)
class TinyImagenetWriter(object):
"""Helper class which writes Tiny Imagenet dataset into TFRecord file."""
def __init__(self, tiny_imagenet_wnid_conveter, imagenet_wnid_converter):
self.tiny_imagenet_wnid_conveter = tiny_imagenet_wnid_conveter
self.imagenet_wnid_converter = imagenet_wnid_converter
def write_tf_record(self,
annotations,
output_file):
"""Generates TFRecord file from given list of annotations."""
with tf.python_io.TFRecordWriter(output_file) as writer:
for image_filename, image_metadata in annotations:
with tf.gfile.Open(image_filename) as f:
image_buffer = f.read()
image_format = get_image_format(image_filename)
features = {
'image/encoded': tf.train.Feature(
bytes_list=tf.train.BytesList(value=[image_buffer])),
'image/format': tf.train.Feature(
bytes_list=tf.train.BytesList(value=[image_format]))
}
if image_metadata:
# bounding box features
features['bbox/xmin'] = tf.train.Feature(
int64_list=tf.train.Int64List(value=[image_metadata.x1]))
features['bbox/ymin'] = tf.train.Feature(
int64_list=tf.train.Int64List(value=[image_metadata.y1]))
features['bbox/xmax'] = tf.train.Feature(
int64_list=tf.train.Int64List(value=[image_metadata.x2]))
features['bbox/ymax'] = tf.train.Feature(
int64_list=tf.train.Int64List(value=[image_metadata.y2]))
# tiny imagenet label, from [0, 200) iterval
tiny_imagenet_label = self.tiny_imagenet_wnid_conveter.to_node_id(
image_metadata.label)
features['label/wnid'] = tf.train.Feature(
bytes_list=tf.train.BytesList(value=image_metadata.label))
features['label/tiny_imagenet'] = tf.train.Feature(
int64_list=tf.train.Int64List(value=[tiny_imagenet_label]))
# full imagenet label, from [1, 1001) interval
if self.imagenet_wnid_converter:
imagenet_label = self.imagenet_wnid_converter.to_node_id(
image_metadata.label)
features['label/imagenet'] = tf.train.Feature(
int64_list=tf.train.Int64List(value=[imagenet_label]))
example = tf.train.Example(features=tf.train.Features(feature=features))
writer.write(example.SerializeToString())
def main(_):
assert FLAGS.input_dir, 'Input directory must be provided'
assert FLAGS.output_dir, 'Output directory must be provided'
# Create WordNet ID conveters for tiny imagenet and possibly for imagenet
tiny_imagenet_wnid_conveter = WnIdToNodeIdConverter(
os.path.join(FLAGS.input_dir, 'wnids.txt'),
background_class=False)
if FLAGS.imagenet_synsets_path:
imagenet_wnid_converter = WnIdToNodeIdConverter(FLAGS.imagenet_synsets_path,
background_class=True)
else:
imagenet_wnid_converter = None
# read tiny imagenet annotations
train_annotations = read_training_annotations(
os.path.join(FLAGS.input_dir, 'train'))
random.shuffle(train_annotations)
val_annotations = read_validation_annotations(
os.path.join(FLAGS.input_dir, 'val'))
test_filenames = read_test_annotations(os.path.join(FLAGS.input_dir, 'test'))
# Generate TFRecord files
writer = TinyImagenetWriter(tiny_imagenet_wnid_conveter,
imagenet_wnid_converter)
tf.logging.info('Converting %d training images', len(train_annotations))
writer.write_tf_record(train_annotations,
os.path.join(FLAGS.output_dir, 'train.tfrecord'))
tf.logging.info('Converting %d validation images ', len(val_annotations))
writer.write_tf_record(val_annotations,
os.path.join(FLAGS.output_dir, 'validation.tfrecord'))
tf.logging.info('Converting %d test images', len(test_filenames))
writer.write_tf_record(test_filenames,
os.path.join(FLAGS.output_dir, 'test.tfrecord'))
tf.logging.info('All files are converted')
if __name__ == '__main__':
app.run(main)
research/adversarial_logit_pairing/train.py deleted 100644 → 0
View file @ 17821c0d
# Copyright 2018 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Program which train models."""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from absl import app
from absl import flags
import tensorflow as tf
import adversarial_attack
import model_lib
from datasets import dataset_factory
FLAGS = flags.FLAGS
flags.DEFINE_integer('max_steps', -1, 'Number of steps to stop at.')
flags.DEFINE_string('output_dir', None,
'Training directory where checkpoints will be saved.')
flags.DEFINE_integer('ps_tasks', 0, 'Number of parameter servers.')
flags.DEFINE_integer('task', 0, 'Task ID for running distributed training.')
flags.DEFINE_string('master', '', 'Tensorflow master.')
flags.DEFINE_string('model_name', 'resnet_v2_50', 'Name of the model.')
flags.DEFINE_string('dataset', 'imagenet',
'Dataset: "tiny_imagenet" or "imagenet".')
flags.DEFINE_integer('dataset_image_size', 64,
'Size of the images in the dataset.')
flags.DEFINE_integer('num_summary_images', 3,
'Number of images to display in Tensorboard.')
flags.DEFINE_integer(
'save_summaries_steps', 100,
'The frequency with which summaries are saved, in steps.')
flags.DEFINE_integer(
'save_summaries_secs', None,
'The frequency with which summaries are saved, in seconds.')
flags.DEFINE_integer(
'save_model_steps', 500,
'The frequency with which the model is saved, in steps.')
flags.DEFINE_string('hparams', '', 'Hyper parameters.')
flags.DEFINE_integer('replicas_to_aggregate', 1,
'Number of gradients to collect before param updates.')
flags.DEFINE_integer('worker_replicas', 1, 'Number of worker replicas.')
flags.DEFINE_float('moving_average_decay', 0.9999,
'The decay to use for the moving average.')
# Flags to control fine tuning
flags.DEFINE_string('finetune_checkpoint_path', None,
'Path to checkpoint for fine tuning. '
'If None then no fine tuning is done.')
flags.DEFINE_string('finetune_exclude_pretrained_scopes', '',
'Variable scopes to exclude when loading checkpoint for '
'fine tuning.')
flags.DEFINE_string('finetune_trainable_scopes', None,
'If set then it defines list of variable scopes for '
'trainable variables.')
def _get_finetuning_init_fn(variable_averages):
"""Returns an init functions, used for fine tuning."""
if not FLAGS.finetune_checkpoint_path:
return None
if tf.train.latest_checkpoint(FLAGS.output_dir):
return None
if tf.gfile.IsDirectory(FLAGS.finetune_checkpoint_path):
checkpoint_path = tf.train.latest_checkpoint(FLAGS.finetune_checkpoint_path)
else:
checkpoint_path = FLAGS.finetune_checkpoint_path
if not checkpoint_path:
tf.logging.warning('Not doing fine tuning, can not find checkpoint in %s',
FLAGS.finetune_checkpoint_path)
return None
tf.logging.info('Fine-tuning from %s', checkpoint_path)
if FLAGS.finetune_exclude_pretrained_scopes:
exclusions = {
scope.strip()
for scope in FLAGS.finetune_exclude_pretrained_scopes.split(',')
}
else:
exclusions = set()
filtered_model_variables = [
v for v in tf.contrib.framework.get_model_variables()
if not any([v.op.name.startswith(e) for e in exclusions])
]
if variable_averages:
variables_to_restore = {}
for v in filtered_model_variables:
# variables_to_restore[variable_averages.average_name(v)] = v
if v in tf.trainable_variables():
variables_to_restore[variable_averages.average_name(v)] = v
else:
variables_to_restore[v.op.name] = v
else:
variables_to_restore = {v.op.name: v for v in filtered_model_variables}
assign_fn = tf.contrib.framework.assign_from_checkpoint_fn(
checkpoint_path,
variables_to_restore)
if assign_fn:
return lambda _, sess: assign_fn(sess)
else:
return None
def main(_):
assert FLAGS.output_dir, '--output_dir has to be provided'
if not tf.gfile.Exists(FLAGS.output_dir):
tf.gfile.MakeDirs(FLAGS.output_dir)
params = model_lib.default_hparams()
params.parse(FLAGS.hparams)
tf.logging.info('User provided hparams: %s', FLAGS.hparams)
tf.logging.info('All hyper parameters: %s', params)
batch_size = params.batch_size
graph = tf.Graph()
with graph.as_default():
with tf.device(tf.train.replica_device_setter(ps_tasks=FLAGS.ps_tasks)):
# dataset
dataset, examples_per_epoch, num_classes, bounds = (
dataset_factory.get_dataset(
FLAGS.dataset,
'train',
batch_size,
FLAGS.dataset_image_size,
is_training=True))
dataset_iterator = dataset.make_one_shot_iterator()
images, labels = dataset_iterator.get_next()
one_hot_labels = tf.one_hot(labels, num_classes)
# set up model
global_step = tf.train.get_or_create_global_step()
model_fn = model_lib.get_model(FLAGS.model_name, num_classes)
if params.train_adv_method == 'clean':
logits = model_fn(images, is_training=True)
adv_examples = None
else:
model_fn_eval_mode = lambda x: model_fn(x, is_training=False)
adv_examples = adversarial_attack.generate_adversarial_examples(
images, bounds, model_fn_eval_mode, params.train_adv_method)
all_examples = tf.concat([images, adv_examples], axis=0)
logits = model_fn(all_examples, is_training=True)
one_hot_labels = tf.concat([one_hot_labels, one_hot_labels], axis=0)
# update trainable variables if fine tuning is used
model_lib.filter_trainable_variables(
FLAGS.finetune_trainable_scopes)
# set up losses
total_loss = tf.losses.softmax_cross_entropy(
onehot_labels=one_hot_labels,
logits=logits,
label_smoothing=params.label_smoothing)
tf.summary.scalar('loss_xent', total_loss)
if params.train_lp_weight > 0:
images1, images2 = tf.split(logits, 2)
loss_lp = tf.losses.mean_squared_error(
images1, images2, weights=params.train_lp_weight)
tf.summary.scalar('loss_lp', loss_lp)
total_loss += loss_lp
if params.weight_decay > 0:
loss_wd = (
params.weight_decay
* tf.add_n([tf.nn.l2_loss(v) for v in tf.trainable_variables()])
)
tf.summary.scalar('loss_wd', loss_wd)
total_loss += loss_wd
# Setup the moving averages:
if FLAGS.moving_average_decay and (FLAGS.moving_average_decay > 0):
with tf.name_scope('moving_average'):
moving_average_variables = tf.contrib.framework.get_model_variables()
variable_averages = tf.train.ExponentialMovingAverage(
FLAGS.moving_average_decay, global_step)
else:
moving_average_variables = None
variable_averages = None
# set up optimizer and training op
learning_rate, steps_per_epoch = model_lib.get_lr_schedule(
params, examples_per_epoch, FLAGS.replicas_to_aggregate)
optimizer = model_lib.get_optimizer(params, learning_rate)
optimizer = tf.train.SyncReplicasOptimizer(
opt=optimizer,
replicas_to_aggregate=FLAGS.replicas_to_aggregate,
total_num_replicas=FLAGS.worker_replicas,
variable_averages=variable_averages,
variables_to_average=moving_average_variables)
train_op = tf.contrib.training.create_train_op(
total_loss, optimizer,
update_ops=tf.get_collection(tf.GraphKeys.UPDATE_OPS))
tf.summary.image('images', images[0:FLAGS.num_summary_images])
if adv_examples is not None:
tf.summary.image('adv_images', adv_examples[0:FLAGS.num_summary_images])
tf.summary.scalar('total_loss', total_loss)
tf.summary.scalar('learning_rate', learning_rate)
tf.summary.scalar('current_epoch',
tf.to_double(global_step) / steps_per_epoch)
# Training
is_chief = FLAGS.task == 0
scaffold = tf.train.Scaffold(
init_fn=_get_finetuning_init_fn(variable_averages))
hooks = [
tf.train.LoggingTensorHook({'total_loss': total_loss,
'global_step': global_step},
every_n_iter=1),
tf.train.NanTensorHook(total_loss),
]
chief_only_hooks = [
tf.train.SummarySaverHook(save_steps=FLAGS.save_summaries_steps,
save_secs=FLAGS.save_summaries_secs,
output_dir=FLAGS.output_dir,
scaffold=scaffold),
tf.train.CheckpointSaverHook(FLAGS.output_dir,
save_steps=FLAGS.save_model_steps,
scaffold=scaffold),
]
if FLAGS.max_steps > 0:
hooks.append(
tf.train.StopAtStepHook(last_step=FLAGS.max_steps))
# hook for sync replica training
hooks.append(optimizer.make_session_run_hook(is_chief))
with tf.train.MonitoredTrainingSession(
master=FLAGS.master,
is_chief=is_chief,
checkpoint_dir=FLAGS.output_dir,
scaffold=scaffold,
hooks=hooks,
chief_only_hooks=chief_only_hooks,
save_checkpoint_secs=None,
save_summaries_steps=None,
save_summaries_secs=None) as session:
while not session.should_stop():
session.run([train_op])
if __name__ == '__main__':
app.run(main)
research/attention_ocr/README.md
View file @ 0cceabfc
## Attention-based Extraction of Structured Information from Street View Imagery
# Attention-based Extraction of Structured Information from Street View Imagery
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/attention-based-extraction-of-structured/optical-character-recognition-on-fsns-test)](https://paperswithcode.com/sota/optical-character-recognition-on-fsns-test?p=attention-based-extraction-of-structured)
[![Paper](http://img.shields.io/badge/paper-arXiv.1704.03549-B3181B.svg)](https://arxiv.org/abs/1704.03549)
......@@ -7,14 +7,20 @@
*A TensorFlow model for real-world image text extraction problems.*
This folder contains the code needed to train a new Attention OCR model on the
[FSNS dataset][FSNS] dataset to transcribe street names in France. You can
also use it to train it on your own data.
[FSNS dataset][FSNS] to transcribe street names in France. You can also train the code on your own data.
More details can be found in our paper:
["Attention-based Extraction of Structured Information from Street View
Imagery"](https://arxiv.org/abs/1704.03549)
## Description
* Paper presents a model based on ConvNets, RNN's and a novel attention mechanism.
Achieves **84.2%** on FSNS beating the previous benchmark (**72.46%**). Also studies
the speed/accuracy tradeoff that results from using CNN feature extractors of
different depths.
## Contacts
Authors
......@@ -22,7 +28,18 @@ Authors
* Zbigniew Wojna (zbigniewwojna@gmail.com)
* Alexander Gorban (gorban@google.com)
Maintainer: Xavier Gibert [@xavigibert](https://github.com/xavigibert)
Maintainer
* Xavier Gibert ([@xavigibert](https://github.com/xavigibert))
## Table of Contents
* [Requirements](https://github.com/tensorflow/models/blob/master/research/attention_ocr/README.md#requirements)
* [Dataset](https://github.com/tensorflow/models/blob/master/research/attention_ocr/README.md#dataset)
* [How to use this code](https://github.com/tensorflow/models/blob/master/research/attention_ocr/README.md#how-to-use-this-code)
* [Using your own image data](https://github.com/tensorflow/models/blob/master/research/attention_ocr/README.md#using-your-own-image-data)
* [How to use a pre-trained model](https://github.com/tensorflow/models/blob/master/research/attention_ocr/README.md#how-to-use-a-pre-trained-model)
* [Disclaimer](https://github.com/tensorflow/models/blob/master/research/attention_ocr/README.md#disclaimer)
## Requirements
......@@ -49,6 +66,42 @@ cd ..
[TF]: https://www.tensorflow.org/install/
[FSNS]: https://github.com/tensorflow/models/tree/master/research/street
## Dataset
The French Street Name Signs (FSNS) dataset is split into subsets,
each of which is composed of multiple files. Note that these datasets
are very large. The approximate sizes are:
* Train: 512 files of 300MB each.
* Validation: 64 files of 40MB each.
* Test: 64 files of 50MB each.
* The datasets download includes a directory `testdata` that contains
some small datasets that are big enough to test that models can
actually learn something.
* Total: around 158GB
The download paths are in the following list:
```
https://download.tensorflow.org/data/fsns-20160927/charset_size=134.txt
https://download.tensorflow.org/data/fsns-20160927/test/test-00000-of-00064
...
https://download.tensorflow.org/data/fsns-20160927/test/test-00063-of-00064
https://download.tensorflow.org/data/fsns-20160927/testdata/arial-32-00000-of-00001
https://download.tensorflow.org/data/fsns-20160927/testdata/fsns-00000-of-00001
https://download.tensorflow.org/data/fsns-20160927/testdata/mnist-sample-00000-of-00001
https://download.tensorflow.org/data/fsns-20160927/testdata/numbers-16-00000-of-00001
https://download.tensorflow.org/data/fsns-20160927/train/train-00000-of-00512
...
https://download.tensorflow.org/data/fsns-20160927/train/train-00511-of-00512
https://download.tensorflow.org/data/fsns-20160927/validation/validation-00000-of-00064
...
https://download.tensorflow.org/data/fsns-20160927/validation/validation-00063-of-00064
```
All URLs are stored in the [research/street](https://github.com/tensorflow/models/tree/master/research/street)
repository in the text file `python/fsns_urls.txt`.
## How to use this code
To run all unit tests:
......@@ -80,7 +133,7 @@ tar xf attention_ocr_2017_08_09.tar.gz
python train.py --checkpoint=model.ckpt-399731
```
## How to use your own image data to train the model
## Using your own image data
You need to define a new dataset. There are two options:
......@@ -166,6 +219,14 @@ implement one in Python or C++.
The recommended way is to use the [Serving infrastructure][serving].
To export to SavedModel format:
```
python model_export.py \
--checkpoint=model.ckpt-399731 \
--export_dir=/tmp/attention_ocr_export
```
Alternatively you can:
1. define a placeholder for images (or use directly an numpy array)
2. [create a graph ](https://github.com/tensorflow/models/blob/master/research/attention_ocr/python/eval.py#L60)
......@@ -188,7 +249,7 @@ other than a one time experiment please use the [TensorFlow Serving][serving].
[1]: https://github.com/tensorflow/tensorflow/blob/aaf7adc/tensorflow/contrib/rnn/python/tools/checkpoint_convert.py
[2]: https://www.tensorflow.org/api_docs/python/tf/contrib/framework/assign_from_checkpoint_fn
[serving]: https://tensorflow.github.io/serving/serving_basic
[serving]: https://www.tensorflow.org/tfx/serving/serving_basic
## Disclaimer
......
research/attention_ocr/python/common_flags.py
View file @ 0cceabfc
......@@ -14,10 +14,10 @@
# ==============================================================================
"""Define flags are common for both train.py and eval.py scripts."""
import logging
import sys
from tensorflow.python.platform import flags
import logging
import datasets
import model
......@@ -35,9 +35,17 @@ logging.basicConfig(
datefmt='%Y-%m-%d %H:%M:%S')
_common_flags_defined = False
def define():
"""Define common flags."""
# yapf: disable
# common_flags.define() may be called multiple times in unit tests.
global _common_flags_defined
if _common_flags_defined:
return
_common_flags_defined = True
flags.DEFINE_integer('batch_size', 32,
'Batch size.')
......@@ -74,7 +82,7 @@ def define():
'the optimizer to use')
flags.DEFINE_float('momentum', 0.9,
'momentum value for the momentum optimizer if used')
'momentum value for the momentum optimizer if used')
flags.DEFINE_bool('use_augment_input', True,
'If True will use image augmentation')
......
research/attention_ocr/python/data_provider.py
View file @ 0cceabfc
......@@ -56,14 +56,14 @@ def augment_image(image):
Returns:
Distorted Tensor image of the same shape.
"""
with tf.variable_scope('AugmentImage'):
with tf.compat.v1.variable_scope('AugmentImage'):
height = image.get_shape().dims[0].value
width = image.get_shape().dims[1].value
# Random crop cut from the street sign image, resized to the same size.
# Assures that the crop is covers at least 0.8 area of the input image.
bbox_begin, bbox_size, _ = tf.image.sample_distorted_bounding_box(
tf.shape(image),
image_size=tf.shape(input=image),
bounding_boxes=tf.zeros([0, 0, 4]),
min_object_covered=0.8,
aspect_ratio_range=[0.8, 1.2],
......@@ -74,7 +74,7 @@ def augment_image(image):
# Randomly chooses one of the 4 interpolation methods
distorted_image = inception_preprocessing.apply_with_random_selector(
distorted_image,
lambda x, method: tf.image.resize_images(x, [height, width], method),
lambda x, method: tf.image.resize(x, [height, width], method),
num_cases=4)
distorted_image.set_shape([height, width, 3])
......@@ -99,9 +99,10 @@ def central_crop(image, crop_size):
Returns:
A tensor of shape [crop_height, crop_width, channels].
"""
with tf.variable_scope('CentralCrop'):
with tf.compat.v1.variable_scope('CentralCrop'):
target_width, target_height = crop_size
image_height, image_width = tf.shape(image)[0], tf.shape(image)[1]
image_height, image_width = tf.shape(
input=image)[0], tf.shape(input=image)[1]
assert_op1 = tf.Assert(
tf.greater_equal(image_height, target_height),
['image_height < target_height', image_height, target_height])
......@@ -129,7 +130,7 @@ def preprocess_image(image, augment=False, central_crop_size=None,
A float32 tensor of shape [H x W x 3] with RGB values in the required
range.
"""
with tf.variable_scope('PreprocessImage'):
with tf.compat.v1.variable_scope('PreprocessImage'):
image = tf.image.convert_image_dtype(image, dtype=tf.float32)
if augment or central_crop_size:
if num_towers == 1:
......@@ -144,9 +145,6 @@ def preprocess_image(image, augment=False, central_crop_size=None,
images = [augment_image(img) for img in images]
image = tf.concat(images, 1)
image = tf.subtract(image, 0.5)
image = tf.multiply(image, 2.5)
return image
......@@ -185,7 +183,7 @@ def get_data(dataset,
image_orig, augment, central_crop_size, num_towers=dataset.num_of_views)
label_one_hot = slim.one_hot_encoding(label, dataset.num_char_classes)
images, images_orig, labels, labels_one_hot = (tf.train.shuffle_batch(
images, images_orig, labels, labels_one_hot = (tf.compat.v1.train.shuffle_batch(
[image, image_orig, label, label_one_hot],
batch_size=batch_size,
num_threads=shuffle_config.num_batching_threads,
......
research/attention_ocr/python/datasets/fsns.py
View file @ 0cceabfc
......@@ -72,7 +72,7 @@ def read_charset(filename, null_character=u'\u2591'):
"""
pattern = re.compile(r'(\d+)\t(.+)')
charset = {}
with tf.gfile.GFile(filename) as f:
with tf.io.gfile.GFile(filename) as f:
for i, line in enumerate(f):
m = pattern.match(line)
if m is None:
......@@ -96,9 +96,9 @@ class _NumOfViewsHandler(slim.tfexample_decoder.ItemHandler):
self._num_of_views = num_of_views
def tensors_to_item(self, keys_to_tensors):
return tf.to_int64(
return tf.cast(
self._num_of_views * keys_to_tensors[self._original_width_key] /
keys_to_tensors[self._width_key])
keys_to_tensors[self._width_key], dtype=tf.int64)
def get_split(split_name, dataset_dir=None, config=None):
......@@ -133,19 +133,19 @@ def get_split(split_name, dataset_dir=None, config=None):
zero = tf.zeros([1], dtype=tf.int64)
keys_to_features = {
'image/encoded':
tf.FixedLenFeature((), tf.string, default_value=''),
tf.io.FixedLenFeature((), tf.string, default_value=''),
'image/format':
tf.FixedLenFeature((), tf.string, default_value='png'),
tf.io.FixedLenFeature((), tf.string, default_value='png'),
'image/width':
tf.FixedLenFeature([1], tf.int64, default_value=zero),
tf.io.FixedLenFeature([1], tf.int64, default_value=zero),
'image/orig_width':
tf.FixedLenFeature([1], tf.int64, default_value=zero),
tf.io.FixedLenFeature([1], tf.int64, default_value=zero),
'image/class':
tf.FixedLenFeature([config['max_sequence_length']], tf.int64),
tf.io.FixedLenFeature([config['max_sequence_length']], tf.int64),
'image/unpadded_class':
tf.VarLenFeature(tf.int64),
tf.io.VarLenFeature(tf.int64),
'image/text':
tf.FixedLenFeature([1], tf.string, default_value=''),
tf.io.FixedLenFeature([1], tf.string, default_value=''),
}
items_to_handlers = {
'image':
......@@ -171,12 +171,14 @@ def get_split(split_name, dataset_dir=None, config=None):
config['splits'][split_name]['pattern'])
return slim.dataset.Dataset(
data_sources=file_pattern,
reader=tf.TFRecordReader,
reader=tf.compat.v1.TFRecordReader,
decoder=decoder,
num_samples=config['splits'][split_name]['size'],
items_to_descriptions=config['items_to_descriptions'],
# additional parameters for convenience.
charset=charset,
charset_file=charset_file,
image_shape=config['image_shape'],
num_char_classes=len(charset),
num_of_views=config['num_of_views'],
max_sequence_length=config['max_sequence_length'],
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment