Merge pull request #3544 from cshallue/master

Add AstroNet to tensorflow/models

research/astronet/.gitignore

+# Ignore all bazel-* symlinks.
+/bazel-*
+
+get_kepler.sh
\ No newline at end of file

research/astronet/README.md

-# Coming Soon!
+# AstroNet: A Neural Network for Identifying Exoplanets in Light Curves
 
-This directory will soon be populated with TensorFlow models and data
-processing code for identifying exoplanets in astrophysical light curves.
+![Transit Animation](docs/transit.gif)
 
-For full details, see the following paper:
+## Contact
 
-*Identifying Exoplanets With Deep Learning: A Five Planet Resonant Chain Around
-Kepler-80 And An Eighth Planet Around Kepler-90*
+Chris Shallue: [@cshallue](https://github.com/cshallue)
 
-Christopher J Shallue and Andrew Vanderburg
+## Background
 
-To appear in the Astronomical Journal
+This directory contains TensorFlow models and data processing code for
+identifying exoplanets in astrophysical light curves. For complete background,
+see [our paper](http://adsabs.harvard.edu/abs/2018AJ....155...94S) in
+*The Astronomical Journal*.
 
-Preprint available at https://www.cfa.harvard.edu/~avanderb/kepler90i.pdf
+For shorter summaries, see:
 
-Contact: Chris Shallue (@cshallue)
+* ["Earth to Exoplanet"](https://www.blog.google/topics/machine-learning/hunting-planets-machine-learning/) on the Google blog
+* [This blog post](https://www.cfa.harvard.edu/~avanderb/page1.html#kepler90) by Andrew Vanderburg
+* [This great article](https://milesobrien.com/artificial-intelligence-gains-intuition-hunting-exoplanets/) by Fedor Kossakovski
+* [NASA's press release](https://www.nasa.gov/press-release/artificial-intelligence-nasa-data-used-to-discover-eighth-planet-circling-distant-star) article
+
+## Citation
+
+If you find this code useful, please cite our paper:
+
+Shallue, C. J., & Vanderburg, A. (2018). Identifying Exoplanets with Deep
+Learning: A Five-planet Resonant Chain around Kepler-80 and an Eighth Planet
+around Kepler-90. *The Astronomical Journal*, 155(2), 94.
+
+Full text available at [*The Astronomical Journal*](http://iopscience.iop.org/article/10.3847/1538-3881/aa9e09/meta).
+
+## Code Directories
+
+[astronet/](astronet/)
+
+* [TensorFlow](https://www.tensorflow.org/) code for:
+  * Downloading and preprocessing Kepler data.
+  * Building different types of neural network classification models.
+  * Training and evaluating a new model.
+  * Using a trained model to generate new predictions.
+
+[light_curve_util/](light_curve_util)
+
+* Utilities for operating on light curves. These include:
+  * Reading Kepler data from `.fits` files.
+  * Applying a median filter to smooth and normalize a light curve.
+  * Phase folding, splitting, removing periodic events, etc.
+* In addition, some C++ implementations of light curve utilities are located in
+[light_curve_util/cc/](light_curve_util/cc).
+
+[third_party/](third_party/)
+
+* Utilities derived from third party code.
+
+
+## Walkthrough
+
+### Install Required Packages
+
+First, ensure that you have installed the following required packages:
+
+* **TensorFlow** ([instructions](https://www.tensorflow.org/install/))
+* **Pandas** ([instructions](http://pandas.pydata.org/pandas-docs/stable/install.html))
+* **NumPy** ([instructions](https://docs.scipy.org/doc/numpy/user/install.html))
+* **AstroPy** ([instructions](http://www.astropy.org/))
+* **PyDl** ([instructions](https://pypi.python.org/pypi/pydl))
+* **Bazel** ([instructions](https://docs.bazel.build/versions/master/install.html))
+* **Abseil Python Common Libraries** ([instructions](https://github.com/abseil/abseil-py))
+    * Optional: only required for unit tests.
+
+### Optional: Run Unit Tests
+
+Verify that all dependencies are satisfied by running the unit tests:
+
+```bash
+bazel test astronet/... light_curve_util/... third_party/...
+```
+
+### Download Kepler Data
+
+A *light curve* is a plot of the brightness of a star over time. We will be
+focusing on light curves produced by the Kepler space telescope, which monitored
+the brightness of 200,000 stars in our milky way galaxy for 4 years. An example
+light curve produced by Kepler is shown below.
+
+![Kepler-934](docs/kepler-943.png)
+
+To train a model to identify planets in Kepler light curves, you will need a
+training set of labeled *Threshold Crossing Events* (TCEs). A TCE is a periodic
+signal that has been detected in a Kepler light curve, and is associated with a
+*period* (the number of days between each occurrence of the detected signal),
+a *duration* (the time taken by each occurrence of the signal), an *epoch* (the
+time of the first observed occurrence of the signal), and possibly additional
+metadata like the signal-to-noise ratio. An example TCE is shown below. The
+labels are ground truth classifications (decided by humans) that indicate which
+TCEs in the training set are actual planets signals and which are caused by
+other phenomena.
+
+![Kepler-934 Transits](docs/kepler-943-transits.png)
+
+You can download the DR24 TCE Table in CSV format from the [NASA Exoplanet
+Archive](https://exoplanetarchive.ipac.caltech.edu/cgi-bin/TblView/nph-tblView?app=ExoTbls&config=q1_q17_dr24_tce). Ensure the following columns are selected:
+
+* `rowid`: Integer ID of the row in the TCE table.
+* `kepid`: Kepler ID of the target star.
+* `tce_plnt_num`: TCE number within the target star.
+* `tce_period`: Period of the detected event, in days.
+* `tce_time0bk`: The time corresponding to the center of the first detected
+      event in Barycentric Julian Day (BJD) minus a constant offset of
+      2,454,833.0 days.
+* `tce_duration`: Duration of the detected event, in hours.
+* `av_training_set`: Autovetter training set label; one of PC (planet candidate),
+      AFP (astrophysical false positive), NTP (non-transiting phenomenon),
+      UNK (unknown).
+
+Next, you will need to download the light curves of the stars corresponding to
+the TCEs in the training set. These are available at the
+[Mikulski Archive for Space Telescopes](https://archive.stsci.edu/). However,
+you almost certainly don't want all of the Kepler data, which consists of almost
+3 million files, takes up over a terabyte of space, and may take several weeks
+to download! To train our model, we only need to download the subset of light
+curves that are associated with TCEs in the DR24 file. To download just those
+light curves, follow these steps:
+
+**NOTE:** Even though we are only downloading a subset of the entire Kepler
+dataset, the files downloaded by the following script take up about **90 GB**.
+
+```bash
+# Filename containing the CSV file of TCEs in the training set.
+TCE_CSV_FILE="${HOME}/astronet/dr24_tce.csv"
+
+# Directory to download Kepler light curves into.
+KEPLER_DATA_DIR="${HOME}/astronet/kepler/"
+
+# Generate a bash script that downloads the Kepler light curves in the training set.
+python astronet/data/generate_download_script.py \
+  --kepler_csv_file=${TCE_CSV_FILE} \
+  --download_dir=${KEPLER_DATA_DIR}
+
+# Run the download script to download Kepler light curves.
+./get_kepler.sh
+```
+
+The final line should read: `Finished downloading 12669 Kepler targets to
+${KEPLER_DATA_DIR}`
+
+Let's explore the downloaded light curve of the Kepler-90 star! Note that Kepler
+light curves are divided into
+[four quarters each year](https://keplerscience.arc.nasa.gov/data-products.html#kepler-data-release-notes), which are separated by the quarterly rolls that the spacecraft
+made to reorient its solar panels. In the downloaded light curves, each `.fits`
+file corresponds to a specific Kepler quarter, but some quarters are divided
+into multiple `.fits` files.
+
+```python
+# Launch iPython (or Python) from the tensorflow_models/astronet/ directory.
+ipython
+
+In[1]:
+from light_curve_util import kepler_io
+import matplotlib.pyplot as plt
+import numpy as np
+
+In[2]:
+KEPLER_DATA_DIR = "/path/to/kepler/"
+KEPLER_ID = 11442793  # Kepler-90.
+
+In[3]:
+# Read the light curve.
+file_names = kepler_io.kepler_filenames(KEPLER_DATA_DIR, KEPLER_ID)
+assert file_names, "Failed to find .fits files in {}".format(KEPLER_DATA_DIR)
+all_time, all_flux = kepler_io.read_kepler_light_curve(file_names)
+print("Read light curve with {} segments".format(len(all_time)))
+
+In[4]:
+# Plot the fourth segment.
+plt.plot(all_time[3], all_flux[3], ".")
+plt.show()
+
+In[5]:
+# Plot all light curve segments. We first divide by the median flux in each
+# segment, because the segments are on different scales.
+for f in all_flux:
+  f /= np.median(f)
+plt.plot(np.concatenate(all_time), np.concatenate(all_flux), ".")
+plt.show()
+```
+The output plots should look something like this:
+
+![Kepler 90 Q4](docs/kep90-q4-raw.png)
+
+![Kepler 90 All](docs/kep90-all.png)
+
+The first plot is a single segment of approximately 20 days. You can see a
+planet transit --- that's Kepler-90 g! Also, notice that the brightness of the
+star is not flat over time --- there is natural variation in the brightness,
+even away from the planet transit.
+
+The second plot is the full light curve over the entire Kepler mission
+(aproximately 4 years). You can easily see two transiting planets by eye ---
+they are Kepler-90 h (the biggest known planet in the system with the deepest
+transits) and Kepler-90 g (the second biggest known planet in the system with
+the second deepest transits).
+
+
+### Process Kepler Data
+
+To train a model to identify exoplanets, you will need to provide TensorFlow
+with training data in
+[TFRecord](https://www.tensorflow.org/programmers_guide/datasets) format. The
+TFRecord format consists of a set of sharded files containing serialized
+`tf.Example` [protocol buffers](https://developers.google.com/protocol-buffers/).
+
+The command below will generate a set of sharded TFRecord files for the TCEs in
+the training set. Each `tf.Example` proto will contain the following light curve
+representations:
+
+* `global_view`: Vector of length 2001: a "global view" of the TCE.
+* `local_view`: Vector of length 201: a "local view" of the TCE.
+
+In addition, each `tf.Example` will contain the value of each column in the
+input TCE CSV file. The columns include:
+
+* `rowid`: Integer ID of the row in the TCE table.
+* `kepid`: Kepler ID of the target star.
+* `tce_plnt_num`: TCE number within the target star.
+* `av_training_set`: Autovetter training set label.
+* `tce_period`: Period of the detected event, in days.
+
+```bash
+# Use Bazel to create executable Python scripts.
+#
+# Alternatively, since all code is pure Python and does not need to be compiled,
+# we could invoke the source scripts with the following addition to PYTHONPATH:
+#     export PYTHONPATH="/path/to/source/dir/:${PYTHONPATH}"
+bazel build astronet/...
+
+# Directory to save output TFRecord files into.
+TFRECORD_DIR="${HOME}/astronet/tfrecord"
+
+# Preprocess light curves into sharded TFRecord files using 5 worker processes.
+bazel-bin/tensorflow/data/generate_input_records \
+  --input_tce_csv_file=${TCE_CSV_FILE} \
+  --kepler_data_dir=${KEPLER_DATA_DIR} \
+  --output_dir=${TFRECORD_DIR} \
+  --num_worker_processes=5
+```
+
+When the script finishes you will find 8 training files, 1 validation file and
+1 test file in `TFRECORD_DIR`. The files will match the patterns
+`train-0000?-of-00008`, `val-00000-of-00001` and `test-00000-of-00001`
+respectively.
+
+Here's a quick description of what the script does. For a full description, see
+Section 3 of [our paper](http://iopscience.iop.org/article/10.3847/1538-3881/aa9e09/meta).
+
+For each light curve, we first fit a normalization spline to remove any
+low-frequency variability (that is, the natural variability in light from star)
+without removing any deviations caused by planets or other objects. For example,
+the following image shows the normalization spline for the segment of Kepler-90
+that we considered above:
+
+![Kepler 90 Q4 Spline](docs/kep90-q4-spline.png)
+
+Next, we divide by the spline to make the star's baseline brightness
+approximately flat. Notice that after normalization the transit of Kepler-90 g
+is still preserved:
+
+![Kepler 90 Q4 Normalized](docs/kep90-q4-normalized.png)
+
+Finally, for each TCE in the input CSV table, we generate two representations of
+the light curve of that star. Both representations are *phase-folded*, which
+means that we combine all periods of the detected TCE into a single curve, with
+the detected event centered.
+
+Let's explore the generated representations of Kepler-90 g in the output.
+
+```python
+# Launch iPython (or Python) from the tensorflow_models/astronet/ directory.
+ipython
+
+In[1]:
+import matplotlib.pyplot as plt
+import numpy as np
+import os.path
+import tensorflow as tf
+
+In[2]:
+KEPLER_ID = 11442793  # Kepler-90
+TFRECORD_DIR = "/path/to/tfrecords/dir"
+
+In[3]:
+# Helper function to find the tf.Example corresponding to a particular TCE.
+def find_tce(kepid, tce_plnt_num, filenames):
+  for filename in filenames:
+    for record in tf.python_io.tf_record_iterator(filename):
+      ex = tf.train.Example.FromString(record)
+      if (ex.features.feature["kepid"].int64_list.value[0] == kepid and
+          ex.features.feature["tce_plnt_num"].int64_list.value[0] == tce_plnt_num):
+        print("Found {}_{} in file {}".format(kepid, tce_plnt_num, filename))
+        return ex
+  raise ValueError("{}_{} not found in files: {}".format(kepid, tce_plnt_num, filenames))
+
+In[4]:
+# Find Kepler-90 g.
+filenames = tf.gfile.Glob(os.path.join(TFRECORD_DIR, "*"))
+assert filenames, "No files found in {}".format(TFRECORD_DIR)
+ex = find_tce(KEPLER_ID, 1, filenames)
+
+In[5]:
+# Plot the global and local views.
+global_view = np.array(ex.features.feature["global_view"].float_list.value)
+local_view = np.array(ex.features.feature["local_view"].float_list.value)
+fig, axes = plt.subplots(1, 2, figsize=(20, 6))
+axes[0].plot(global_view, ".")
+axes[1].plot(local_view, ".")
+plt.show()
+```
+
+The output should look something like this:
+
+![Kepler 90 g Processed](docs/kep90h-localglobal.png)
+
+### Train an AstroNet Model
+
+The [astronet](astronet/) directory contains several types of neural
+network architecture and various configuration options. To train a convolutional
+neural network to classify Kepler TCEs as either "planet" or "not planet",
+using the best configuration from
+[our paper](http://iopscience.iop.org/article/10.3847/1538-3881/aa9e09/meta),
+run the following training script:
+
+```bash
+# Directory to save model checkpoints into.
+MODEL_DIR="${HOME}/astronet/model/"
+
+# Run the training script.
+bazel-bin/astronet/train \
+  --model=AstroCNNModel \
+  --config_name=local_global \
+  --train_files=${TFRECORD_DIR}/train* \
+  --eval_files=${TFRECORD_DIR}/val* \
+  --model_dir=${MODEL_DIR}
+```
+
+Optionally, you can also run a [TensorBoard](https://www.tensorflow.org/programmers_guide/summaries_and_tensorboard)
+server in a separate process for real-time
+monitoring of training progress and evaluation metrics.
+
+```bash
+# Launch TensorBoard server.
+tensorboard --logdir ${MODEL_DIR}
+```
+
+The TensorBoard server will show a page like this:
+
+![TensorBoard](docs/tensorboard.png)
+
+### Evaluate an AstroNet Model
+
+Run the following command to evaluate a model on the test set. The result will
+be printed on the screen, and a summary file will also be written to the model
+directory, which will be visible in TensorBoard.
+
+```bash
+# Run the evaluation script.
+bazel-bin/astronet/evaluate \
+  --model=AstroCNNModel \
+  --config_name=local_global \
+  --eval_files=${TFRECORD_DIR}/test* \
+  --model_dir=${MODEL_DIR}
+```
+
+The output should look something like this:
+
+```bash
+INFO:tensorflow:Saving dict for global step 10000: accuracy/accuracy = 0.9625159, accuracy/num_correct = 1515.0, auc = 0.988882, confusion_matrix/false_negatives = 10.0, confusion_matrix/false_positives = 49.0, confusion_matrix/true_negatives = 1165.0, confusion_matrix/true_positives = 350.0, global_step = 10000, loss = 0.112445444, losses/weighted_cross_entropy = 0.11295206, num_examples = 1574.
+```
+
+### Make Predictions
+
+Suppose you detect a weak TCE in the light curve of the Kepler-90 star, with
+period 14.44912 days, duration 2.70408 hours (0.11267 days) beginning 2.2 days
+after 12:00 on 1/1/2009 (the year the Kepler telescope launched). To run this
+TCE though your trained model, execute the following command:
+
+```bash
+# Generate a prediction for a new TCE.
+bazel-bin/astronet/predict \
+  --model=AstroCNNModel \
+  --config_name=local_global \
+  --model_dir=${MODEL_DIR} \
+  --kepler_data_dir=${KEPLER_DATA_DIR} \
+  --kepler_id=11442793 \
+  --period=14.44912 \
+  --t0=2.2 \
+  --duration=0.11267 \
+  --output_image_file="${HOME}/astronet/kepler-90i.png"
+```
+
+The output should look like this:
+
+```Prediction: 0.9480018```
+
+This means that the model is about 95% confident that the input TCE is a planet.
+Indeed, this TCE is
+[Kepler-90 i](https://www.nasa.gov/press-release/artificial-intelligence-nasa-data-used-to-discover-eighth-planet-circling-distant-star),
+which is the eighth planet discovered around the Kepler-90 star!
+
+In addition to the output prediction, the script will also produce a plot of the
+input representations. For Kepler-90 i, the plot should look something like
+this:
+
+![Kepler 90 h Processed](docs/kep90i-localglobal.png)

research/astronet/WORKSPACE

+# Abseil C++ libraries.
+http_archive(
+     name = "com_google_absl",
+     urls = ["https://github.com/abseil/abseil-cpp/archive/master.zip"],
+     strip_prefix = "abseil-cpp-master",
+)
+
+# GoogleTest/GoogleMock framework. Used by C++ unit-tests.
+http_archive(
+     name = "com_google_googletest",
+     urls = ["https://github.com/google/googletest/archive/master.zip"],
+     strip_prefix = "googletest-master",
+)

research/astronet/astronet/BUILD

+package(default_visibility = ["//visibility:public"])
+
+licenses(["notice"])  # Apache 2.0
+
+py_library(
+    name = "models",
+    srcs = ["models.py"],
+    srcs_version = "PY2AND3",
+    deps = [
+        "//astronet/astro_cnn_model",
+        "//astronet/astro_cnn_model:configurations",
+        "//astronet/astro_fc_model",
+        "//astronet/astro_fc_model:configurations",
+        "//astronet/astro_model",
+        "//astronet/astro_model:configurations",
+        "//astronet/util:configdict",
+    ],
+)
+
+py_binary(
+    name = "train",
+    srcs = ["train.py"],
+    srcs_version = "PY2AND3",
+    deps = [
+        ":models",
+        "//astronet/util:config_util",
+        "//astronet/util:configdict",
+        "//astronet/util:estimator_util",
+    ],
+)
+
+py_binary(
+    name = "evaluate",
+    srcs = ["evaluate.py"],
+    srcs_version = "PY2AND3",
+    deps = [
+        ":models",
+        "//astronet/util:config_util",
+        "//astronet/util:configdict",
+        "//astronet/util:estimator_util",
+    ],
+)
+
+py_binary(
+    name = "predict",
+    srcs = ["predict.py"],
+    srcs_version = "PY2AND3",
+    deps = [
+        ":models",
+        "//astronet/data:preprocess",
+        "//astronet/util:config_util",
+        "//astronet/util:configdict",
+        "//astronet/util:estimator_util",
+    ],
+)

research/astronet/astronet/__init__.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+

research/astronet/astronet/astro_cnn_model/BUILD

+package(default_visibility = ["//visibility:public"])
+
+licenses(["notice"])  # Apache 2.0
+
+py_library(
+    name = "configurations",
+    srcs = ["configurations.py"],
+    srcs_version = "PY2AND3",
+    deps = [
+        "//astronet/astro_model:configurations",
+    ],
+)
+
+py_library(
+    name = "astro_cnn_model",
+    srcs = [
+        "astro_cnn_model.py",
+    ],
+    srcs_version = "PY2AND3",
+    deps = ["//astronet/astro_model"],
+)
+
+py_test(
+    name = "astro_cnn_model_test",
+    size = "small",
+    srcs = [
+        "astro_cnn_model_test.py",
+    ],
+    srcs_version = "PY2AND3",
+    deps = [
+        ":astro_cnn_model",
+        ":configurations",
+        "//astronet/ops:input_ops",
+        "//astronet/ops:testing",
+        "//astronet/util:configdict",
+    ],
+)

research/astronet/astronet/astro_cnn_model/__init__.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+

research/astronet/astronet/astro_cnn_model/astro_cnn_model.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""A model for classifying light curves using a convolutional neural network.
+
+See the base class (in astro_model.py) for a description of the general
+framework of AstroModel and its subclasses.
+
+The architecture of this model is:
+
+
+                                     predictions
+                                          ^
+                                          |
+                                       logits
+                                          ^
+                                          |
+                                (fully connected layers)
+                                          ^
+                                          |
+                                   pre_logits_concat
+                                          ^
+                                          |
+                                    (concatenate)
+
+              ^                           ^                          ^
+              |                           |                          |
+   (convolutional blocks 1)  (convolutional blocks 2)   ...          |
+              ^                           ^                          |
+              |                           |                          |
+     time_series_feature_1     time_series_feature_2    ...     aux_features
+"""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import tensorflow as tf
+
+from astronet.astro_model import astro_model
+
+
+class AstroCNNModel(astro_model.AstroModel):
+  """A model for classifying light curves using a convolutional neural net."""
+
+  def __init__(self, features, labels, hparams, mode):
+    """Basic setup. The actual TensorFlow graph is constructed in build().
+
+    Args:
+      features: A dictionary containing "time_series_features" and
+          "aux_features", each of which is a dictionary of named input Tensors.
+          All features have dtype float32 and shape [batch_size, length].
+      labels: An int64 Tensor with shape [batch_size]. May be None if mode is
+          tf.estimator.ModeKeys.PREDICT.
+      hparams: A ConfigDict of hyperparameters for building the model.
+      mode: A tf.estimator.ModeKeys to specify whether the graph should be built
+          for training, evaluation or prediction.
+
+    Raises:
+      ValueError: If mode is invalid.
+    """
+    super(AstroCNNModel, self).__init__(features, labels, hparams, mode)
+
+  def _build_cnn_layers(self, inputs, hparams, scope="cnn"):
+    """Builds convolutional layers.
+
+    The layers are defined by convolutional blocks with pooling between blocks
+    (but not within blocks). Within a block, all layers have the same number of
+    filters, which is a constant multiple of the number of filters in the
+    previous block. The kernel size is fixed throughout.
+
+    Args:
+      inputs: A Tensor of shape [batch_size, length].
+      hparams: Object containing CNN hyperparameters.
+      scope: Name of the variable scope.
+
+    Returns:
+      A Tensor of shape [batch_size, output_size], where the output size depends
+      on the input size, kernel size, number of filters, number of layers,
+      convolution padding type and pooling.
+    """
+    with tf.variable_scope(scope):
+      net = tf.expand_dims(inputs, -1)  # [batch, length, channels]
+      for i in range(hparams.cnn_num_blocks):
+        num_filters = int(hparams.cnn_initial_num_filters *
+                          hparams.cnn_block_filter_factor**i)
+        with tf.variable_scope("block_%d" % (i + 1)):
+          for j in range(hparams.cnn_block_size):
+            net = tf.layers.conv1d(
+                inputs=net,
+                filters=num_filters,
+                kernel_size=int(hparams.cnn_kernel_size),
+                padding=hparams.convolution_padding,
+                activation=tf.nn.relu,
+                name="conv_%d" % (j + 1))
+
+          if hparams.pool_size > 1:  # pool_size 0 or 1 denotes no pooling
+            net = tf.layers.max_pooling1d(
+                inputs=net,
+                pool_size=int(hparams.pool_size),
+                strides=int(hparams.pool_strides),
+                name="pool")
+
+      # Flatten.
+      net.get_shape().assert_has_rank(3)
+      net_shape = net.get_shape().as_list()
+      output_dim = net_shape[1] * net_shape[2]
+      net = tf.reshape(net, [-1, output_dim], name="flatten")
+
+    return net
+
+  def build_time_series_hidden_layers(self):
+    """Builds hidden layers for the time series features.
+
+    Inputs:
+      self.time_series_features
+
+    Outputs:
+      self.time_series_hidden_layers
+    """
+    time_series_hidden_layers = {}
+    for name, time_series in self.time_series_features.iteritems():
+      time_series_hidden_layers[name] = self._build_cnn_layers(
+          inputs=time_series,
+          hparams=self.hparams.time_series_hidden[name],
+          scope=name + "_hidden")
+
+    self.time_series_hidden_layers = time_series_hidden_layers

research/astronet/astronet/astro_cnn_model/astro_cnn_model_test.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Tests for astro_cnn_model.AstroCNNModel."""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import numpy as np
+import tensorflow as tf
+
+from astronet.astro_cnn_model import astro_cnn_model
+from astronet.astro_cnn_model import configurations
+from astronet.ops import input_ops
+from astronet.ops import testing
+from astronet.util import configdict
+
+
+class AstroCNNModelTest(tf.test.TestCase):
+
+  def assertShapeEquals(self, shape, tensor_or_array):
+    """Asserts that a Tensor or Numpy array has the expected shape.
+
+    Args:
+      shape: Numpy array or anything that can be converted to one.
+      tensor_or_array: tf.Tensor, tf.Variable, Numpy array or anything that can
+          be converted to one.
+    """
+    if isinstance(tensor_or_array, (np.ndarray, np.generic)):
+      self.assertAllEqual(shape, tensor_or_array.shape)
+    elif isinstance(tensor_or_array, (tf.Tensor, tf.Variable)):
+      self.assertAllEqual(shape, tensor_or_array.shape.as_list())
+    else:
+      raise TypeError("tensor_or_array must be a Tensor or Numpy ndarray")
+
+  def testOneTimeSeriesFeature(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 20,
+            "is_time_series": True,
+        }
+    }
+    hidden_spec = {
+        "time_feature_1": {
+            "cnn_num_blocks": 2,
+            "cnn_block_size": 2,
+            "cnn_initial_num_filters": 4,
+            "cnn_block_filter_factor": 1.5,
+            "cnn_kernel_size": 3,
+            "convolution_padding": "same",
+            "pool_size": 2,
+            "pool_strides": 2,
+        }
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config["hparams"]["time_series_hidden"] = hidden_spec
+    config = configdict.ConfigDict(config)
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_cnn_model.AstroCNNModel(features, labels, config.hparams,
+                                          tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate Tensor shapes.
+    block_1_conv_1 = testing.get_variable_by_name(
+        "time_feature_1_hidden/block_1/conv_1/kernel")
+    self.assertShapeEquals((3, 1, 4), block_1_conv_1)
+
+    block_1_conv_2 = testing.get_variable_by_name(
+        "time_feature_1_hidden/block_1/conv_2/kernel")
+    self.assertShapeEquals((3, 4, 4), block_1_conv_2)
+
+    block_2_conv_1 = testing.get_variable_by_name(
+        "time_feature_1_hidden/block_2/conv_1/kernel")
+    self.assertShapeEquals((3, 4, 6), block_2_conv_1)
+
+    block_2_conv_2 = testing.get_variable_by_name(
+        "time_feature_1_hidden/block_2/conv_2/kernel")
+    self.assertShapeEquals((3, 6, 6), block_2_conv_2)
+
+    self.assertItemsEqual(["time_feature_1"],
+                          model.time_series_hidden_layers.keys())
+    self.assertShapeEquals((None, 30),
+                           model.time_series_hidden_layers["time_feature_1"])
+    self.assertEqual(len(model.aux_hidden_layers), 0)
+    self.assertIs(model.time_series_hidden_layers["time_feature_1"],
+                  model.pre_logits_concat)
+
+    # Execute the TensorFlow graph.
+    scaffold = tf.train.Scaffold()
+    scaffold.finalize()
+    with self.test_session() as sess:
+      sess.run([scaffold.init_op, scaffold.local_init_op])
+      step = sess.run(model.global_step)
+      self.assertEqual(0, step)
+
+      # Fetch predictions.
+      features = testing.fake_features(feature_spec, batch_size=16)
+      labels = testing.fake_labels(config.hparams.output_dim, batch_size=16)
+      feed_dict = input_ops.prepare_feed_dict(model, features, labels)
+      predictions = sess.run(model.predictions, feed_dict=feed_dict)
+      self.assertShapeEquals((16, 1), predictions)
+
+  def testTwoTimeSeriesFeatures(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 20,
+            "is_time_series": True,
+        },
+        "time_feature_2": {
+            "length": 5,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+    }
+    hidden_spec = {
+        "time_feature_1": {
+            "cnn_num_blocks": 2,
+            "cnn_block_size": 2,
+            "cnn_initial_num_filters": 4,
+            "cnn_block_filter_factor": 1.5,
+            "cnn_kernel_size": 3,
+            "convolution_padding": "same",
+            "pool_size": 2,
+            "pool_strides": 2,
+        },
+        "time_feature_2": {
+            "cnn_num_blocks": 1,
+            "cnn_block_size": 1,
+            "cnn_initial_num_filters": 5,
+            "cnn_block_filter_factor": 1,
+            "cnn_kernel_size": 2,
+            "convolution_padding": "same",
+            "pool_size": 0,
+            "pool_strides": 0,
+        }
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config["hparams"]["time_series_hidden"] = hidden_spec
+    config = configdict.ConfigDict(config)
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_cnn_model.AstroCNNModel(features, labels, config.hparams,
+                                          tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate Tensor shapes.
+    feature_1_block_1_conv_1 = testing.get_variable_by_name(
+        "time_feature_1_hidden/block_1/conv_1/kernel")
+    self.assertShapeEquals((3, 1, 4), feature_1_block_1_conv_1)
+
+    feature_1_block_1_conv_2 = testing.get_variable_by_name(
+        "time_feature_1_hidden/block_1/conv_2/kernel")
+    self.assertShapeEquals((3, 4, 4), feature_1_block_1_conv_2)
+
+    feature_1_block_2_conv_1 = testing.get_variable_by_name(
+        "time_feature_1_hidden/block_2/conv_1/kernel")
+    self.assertShapeEquals((3, 4, 6), feature_1_block_2_conv_1)
+
+    feature_1_block_2_conv_2 = testing.get_variable_by_name(
+        "time_feature_1_hidden/block_2/conv_2/kernel")
+    self.assertShapeEquals((3, 6, 6), feature_1_block_2_conv_2)
+
+    feature_2_block_1_conv_1 = testing.get_variable_by_name(
+        "time_feature_2_hidden/block_1/conv_1/kernel")
+    self.assertShapeEquals((2, 1, 5), feature_2_block_1_conv_1)
+
+    self.assertItemsEqual(["time_feature_1", "time_feature_2"],
+                          model.time_series_hidden_layers.keys())
+    self.assertShapeEquals((None, 30),
+                           model.time_series_hidden_layers["time_feature_1"])
+    self.assertShapeEquals((None, 25),
+                           model.time_series_hidden_layers["time_feature_2"])
+    self.assertItemsEqual(["aux_feature_1"], model.aux_hidden_layers.keys())
+    self.assertIs(model.aux_features["aux_feature_1"],
+                  model.aux_hidden_layers["aux_feature_1"])
+    self.assertShapeEquals((None, 56), model.pre_logits_concat)
+
+    # Execute the TensorFlow graph.
+    scaffold = tf.train.Scaffold()
+    scaffold.finalize()
+    with self.test_session() as sess:
+      sess.run([scaffold.init_op, scaffold.local_init_op])
+      step = sess.run(model.global_step)
+      self.assertEqual(0, step)
+
+      # Fetch predictions.
+      features = testing.fake_features(feature_spec, batch_size=16)
+      labels = testing.fake_labels(config.hparams.output_dim, batch_size=16)
+      feed_dict = input_ops.prepare_feed_dict(model, features, labels)
+      predictions = sess.run(model.predictions, feed_dict=feed_dict)
+      self.assertShapeEquals((16, 1), predictions)
+
+
+if __name__ == "__main__":
+  tf.test.main()

research/astronet/astronet/astro_cnn_model/configurations.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Configurations for model building, training and evaluation.
+
+Available configurations:
+  * base: One time series feature per input example. Default is "global_view".
+  * local_global: Two time series features per input example.
+      - A "global" view of the entire orbital period.
+      - A "local" zoomed-in view of the transit event.
+"""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+from astronet.astro_model import configurations as parent_configs
+
+
+def base():
+  """Base configuration for a CNN model with a single global view."""
+  config = parent_configs.base()
+
+  # Add configuration for the convolutional layers of the global_view feature.
+  config["hparams"]["time_series_hidden"] = {
+      "global_view": {
+          "cnn_num_blocks": 5,
+          "cnn_block_size": 2,
+          "cnn_initial_num_filters": 16,
+          "cnn_block_filter_factor": 2,
+          "cnn_kernel_size": 5,
+          "convolution_padding": "same",
+          "pool_size": 5,
+          "pool_strides": 2,
+      },
+  }
+  config["hparams"]["num_pre_logits_hidden_layers"] = 4
+  config["hparams"]["pre_logits_hidden_layer_size"] = 1024
+  return config
+
+
+def local_global():
+  """Base configuration for a CNN model with separate local/global views."""
+  config = parent_configs.base()
+
+  # Override the model features to be local_view and global_view time series.
+  config["inputs"]["features"] = {
+      "local_view": {
+          "length": 201,
+          "is_time_series": True,
+      },
+      "global_view": {
+          "length": 2001,
+          "is_time_series": True,
+      },
+  }
+
+  # Add configurations for the convolutional layers of time series features.
+  config["hparams"]["time_series_hidden"] = {
+      "local_view": {
+          "cnn_num_blocks": 2,
+          "cnn_block_size": 2,
+          "cnn_initial_num_filters": 16,
+          "cnn_block_filter_factor": 2,
+          "cnn_kernel_size": 5,
+          "convolution_padding": "same",
+          "pool_size": 7,
+          "pool_strides": 2,
+      },
+      "global_view": {
+          "cnn_num_blocks": 5,
+          "cnn_block_size": 2,
+          "cnn_initial_num_filters": 16,
+          "cnn_block_filter_factor": 2,
+          "cnn_kernel_size": 5,
+          "convolution_padding": "same",
+          "pool_size": 5,
+          "pool_strides": 2,
+      },
+  }
+  config["hparams"]["num_pre_logits_hidden_layers"] = 4
+  config["hparams"]["pre_logits_hidden_layer_size"] = 512
+  return config

research/astronet/astronet/astro_fc_model/BUILD

+package(default_visibility = ["//visibility:public"])
+
+licenses(["notice"])  # Apache 2.0
+
+py_library(
+    name = "configurations",
+    srcs = ["configurations.py"],
+    srcs_version = "PY2AND3",
+    deps = [
+        "//astronet/astro_model:configurations",
+    ],
+)
+
+py_library(
+    name = "astro_fc_model",
+    srcs = [
+        "astro_fc_model.py",
+    ],
+    srcs_version = "PY2AND3",
+    deps = ["//astronet/astro_model"],
+)
+
+py_test(
+    name = "astro_fc_model_test",
+    size = "small",
+    srcs = [
+        "astro_fc_model_test.py",
+    ],
+    srcs_version = "PY2AND3",
+    deps = [
+        ":astro_fc_model",
+        ":configurations",
+        "//astronet/ops:input_ops",
+        "//astronet/ops:testing",
+        "//astronet/util:configdict",
+    ],
+)

research/astronet/astronet/astro_fc_model/__init__.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+

research/astronet/astronet/astro_fc_model/astro_fc_model.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""A model for classifying light curves using (locally) fully connected layers.
+
+Note that the first layer of each fully connected stack is optionally
+implemented as a convolution with a wide kernel followed by pooling. This causes
+invariance to small translations.
+
+See the base class (in astro_model.py) for a description of the general
+framework of AstroModel and its subclasses.
+
+The architecture of this model is:
+
+
+                                     predictions
+                                          ^
+                                          |
+                                       logits
+                                          ^
+                                          |
+                                (fully connected layers)
+                                          ^
+                                          |
+                                   pre_logits_concat
+                                          ^
+                                          |
+                                    (concatenate)
+
+              ^                           ^                          ^
+              |                           |                          |
+ (locally fully connected 1)  (locally fully connected 2)  ...       |
+              ^                           ^                          |
+              |                           |                          |
+     time_series_feature_1      time_series_feature_2      ...  aux_features
+"""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import tensorflow as tf
+
+from astronet.astro_model import astro_model
+
+
+class AstroFCModel(astro_model.AstroModel):
+  """A model for classifying light curves using fully connected layers."""
+
+  def __init__(self, features, labels, hparams, mode):
+    """Basic setup. The actual TensorFlow graph is constructed in build().
+
+    Args:
+      features: A dictionary containing "time_series_features" and
+          "aux_features", each of which is a dictionary of named input Tensors.
+          All features have dtype float32 and shape [batch_size, length].
+      labels: An int64 Tensor with shape [batch_size]. May be None if mode is
+          tf.estimator.ModeKeys.PREDICT.
+      hparams: A ConfigDict of hyperparameters for building the model.
+      mode: A tf.estimator.ModeKeys to specify whether the graph should be built
+          for training, evaluation or prediction.
+
+    Raises:
+      ValueError: If mode is invalid.
+    """
+    super(AstroFCModel, self).__init__(features, labels, hparams, mode)
+
+  def _build_local_fc_layers(self, inputs, hparams, scope):
+    """Builds locally fully connected layers.
+
+    Note that the first layer of the fully connected stack is optionally
+    implemented as a convolution with a wide kernel followed by pooling. This
+    makes the fully connected stack invariant to small translations of its
+    input.
+
+    Args:
+      inputs: A Tensor of shape [batch_size, length].
+      hparams: Object containing hyperparameters.
+      scope: Name of the variable scope.
+
+    Returns:
+      A Tensor of shape [batch_size, hparams.local_layer_size].
+
+    Raises:
+      ValueError: If hparams.pooling_type is unrecognized.
+    """
+    if hparams.num_local_layers == 0:
+      return inputs
+
+    net = inputs
+    with tf.variable_scope(scope):
+      # First layer is optionally implemented as a wide convolution for
+      # invariance to small translations.
+      if hparams.translation_delta > 0:
+        kernel_size = inputs.shape.as_list()[1] - 2 * hparams.translation_delta
+        net = tf.expand_dims(net, -1)  # [batch, length, channels=1]
+        net = tf.layers.conv1d(
+            inputs=net,
+            filters=hparams.local_layer_size,
+            kernel_size=kernel_size,
+            padding="valid",
+            activation=tf.nn.relu,
+            name="conv1d")
+
+        # net is [batch, length, num_filters], where length = 1 +
+        # 2 * translation_delta. Pool along the length dimension.
+        if hparams.pooling_type == "max":
+          net = tf.reduce_max(net, axis=1, name="max_pool")
+        elif hparams.pooling_type == "avg":
+          net = tf.reduce_mean(net, axis=1, name="avg_pool")
+        else:
+          raise ValueError(
+              "Unrecognized pooling_type: %s" % hparams.pooling_type)
+
+        remaining_layers = hparams.num_local_layers - 1
+      else:
+        remaining_layers = hparams.num_local_layers
+
+      # Remaining fully connected layers.
+      for i in range(remaining_layers):
+        net = tf.contrib.layers.fully_connected(
+            inputs=net,
+            num_outputs=hparams.local_layer_size,
+            activation_fn=tf.nn.relu,
+            scope="fully_connected_%d" % (i + 1))
+
+        if hparams.dropout_rate > 0:
+          net = tf.layers.dropout(
+              net, hparams.dropout_rate, training=self.is_training)
+
+    return net
+
+  def build_time_series_hidden_layers(self):
+    """Builds hidden layers for the time series features.
+
+    Inputs:
+      self.time_series_features
+
+    Outputs:
+      self.time_series_hidden_layers
+    """
+    time_series_hidden_layers = {}
+    for name, time_series in self.time_series_features.iteritems():
+      time_series_hidden_layers[name] = self._build_local_fc_layers(
+          inputs=time_series,
+          hparams=self.hparams.time_series_hidden[name],
+          scope=name + "_hidden")
+
+    self.time_series_hidden_layers = time_series_hidden_layers

research/astronet/astronet/astro_fc_model/astro_fc_model_test.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Tests for astro_fc_model.AstroFCModel."""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import numpy as np
+import tensorflow as tf
+
+from astronet.astro_fc_model import astro_fc_model
+from astronet.astro_fc_model import configurations
+from astronet.ops import input_ops
+from astronet.ops import testing
+from astronet.util import configdict
+
+
+class AstroFCModelTest(tf.test.TestCase):
+
+  def assertShapeEquals(self, shape, tensor_or_array):
+    """Asserts that a Tensor or Numpy array has the expected shape.
+
+    Args:
+      shape: Numpy array or anything that can be converted to one.
+      tensor_or_array: tf.Tensor, tf.Variable, Numpy array or anything that can
+          be converted to one.
+    """
+    if isinstance(tensor_or_array, (np.ndarray, np.generic)):
+      self.assertAllEqual(shape, tensor_or_array.shape)
+    elif isinstance(tensor_or_array, (tf.Tensor, tf.Variable)):
+      self.assertAllEqual(shape, tensor_or_array.shape.as_list())
+    else:
+      raise TypeError("tensor_or_array must be a Tensor or Numpy ndarray")
+
+  def testOneTimeSeriesFeature(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 14,
+            "is_time_series": True,
+        }
+    }
+    hidden_spec = {
+        "time_feature_1": {
+            "num_local_layers": 2,
+            "local_layer_size": 20,
+            "translation_delta": 2,
+            "pooling_type": "max",
+            "dropout_rate": 0.5,
+        }
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config["hparams"]["time_series_hidden"] = hidden_spec
+    config = configdict.ConfigDict(config)
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_fc_model.AstroFCModel(features, labels, config.hparams,
+                                        tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate Tensor shapes.
+    conv = testing.get_variable_by_name("time_feature_1_hidden/conv1d/kernel")
+    self.assertShapeEquals((10, 1, 20), conv)
+
+    fc_1 = testing.get_variable_by_name(
+        "time_feature_1_hidden/fully_connected_1/weights")
+    self.assertShapeEquals((20, 20), fc_1)
+
+    self.assertItemsEqual(["time_feature_1"],
+                          model.time_series_hidden_layers.keys())
+    self.assertShapeEquals((None, 20),
+                           model.time_series_hidden_layers["time_feature_1"])
+    self.assertEqual(len(model.aux_hidden_layers), 0)
+    self.assertIs(model.time_series_hidden_layers["time_feature_1"],
+                  model.pre_logits_concat)
+
+    # Execute the TensorFlow graph.
+    scaffold = tf.train.Scaffold()
+    scaffold.finalize()
+    with self.test_session() as sess:
+      sess.run([scaffold.init_op, scaffold.local_init_op])
+      step = sess.run(model.global_step)
+      self.assertEqual(0, step)
+
+      # Fetch predictions.
+      features = testing.fake_features(feature_spec, batch_size=16)
+      labels = testing.fake_labels(config.hparams.output_dim, batch_size=16)
+      feed_dict = input_ops.prepare_feed_dict(model, features, labels)
+      predictions = sess.run(model.predictions, feed_dict=feed_dict)
+      self.assertShapeEquals((16, 1), predictions)
+
+  def testTwoTimeSeriesFeatures(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 20,
+            "is_time_series": True,
+        },
+        "time_feature_2": {
+            "length": 5,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+    }
+    hidden_spec = {
+        "time_feature_1": {
+            "num_local_layers": 1,
+            "local_layer_size": 20,
+            "translation_delta": 1,
+            "pooling_type": "max",
+            "dropout_rate": 0.5,
+        },
+        "time_feature_2": {
+            "num_local_layers": 2,
+            "local_layer_size": 7,
+            "translation_delta": 0,
+            "dropout_rate": 0,
+        }
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config["hparams"]["time_series_hidden"] = hidden_spec
+    config = configdict.ConfigDict(config)
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_fc_model.AstroFCModel(features, labels, config.hparams,
+                                        tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate Tensor shapes.
+    conv = testing.get_variable_by_name("time_feature_1_hidden/conv1d/kernel")
+    self.assertShapeEquals((18, 1, 20), conv)
+
+    fc_1 = testing.get_variable_by_name(
+        "time_feature_2_hidden/fully_connected_1/weights")
+    self.assertShapeEquals((5, 7), fc_1)
+
+    fc_2 = testing.get_variable_by_name(
+        "time_feature_2_hidden/fully_connected_2/weights")
+    self.assertShapeEquals((7, 7), fc_2)
+
+    self.assertItemsEqual(["time_feature_1", "time_feature_2"],
+                          model.time_series_hidden_layers.keys())
+    self.assertShapeEquals((None, 20),
+                           model.time_series_hidden_layers["time_feature_1"])
+    self.assertShapeEquals((None, 7),
+                           model.time_series_hidden_layers["time_feature_2"])
+    self.assertItemsEqual(["aux_feature_1"], model.aux_hidden_layers.keys())
+    self.assertIs(model.aux_features["aux_feature_1"],
+                  model.aux_hidden_layers["aux_feature_1"])
+    self.assertShapeEquals((None, 28), model.pre_logits_concat)
+
+    # Execute the TensorFlow graph.
+    scaffold = tf.train.Scaffold()
+    scaffold.finalize()
+    with self.test_session() as sess:
+      sess.run([scaffold.init_op, scaffold.local_init_op])
+      step = sess.run(model.global_step)
+      self.assertEqual(0, step)
+
+      # Fetch predictions.
+      features = testing.fake_features(feature_spec, batch_size=16)
+      labels = testing.fake_labels(config.hparams.output_dim, batch_size=16)
+      feed_dict = input_ops.prepare_feed_dict(model, features, labels)
+      predictions = sess.run(model.predictions, feed_dict=feed_dict)
+      self.assertShapeEquals((16, 1), predictions)
+
+
+if __name__ == "__main__":
+  tf.test.main()

research/astronet/astronet/astro_fc_model/configurations.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Configurations for model building, training and evaluation.
+
+Available configurations:
+  * base: One time series feature per input example. Default is "global_view".
+  * local_global: Two time series features per input example.
+      - A "global" view of the entire orbital period.
+      - A "local" zoomed-in view of the transit event.
+"""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+from astronet.astro_model import configurations as parent_configs
+
+
+def base():
+  """Base config for a fully connected model with a single global view."""
+  config = parent_configs.base()
+
+  # Add configuration for the fully-connected layers of the global_view feature.
+  config["hparams"]["time_series_hidden"] = {
+      "global_view": {
+          "num_local_layers": 0,
+          "local_layer_size": 128,
+
+          # If > 0, the first layer is implemented as a wide convolutional layer
+          # for invariance to small translations.
+          "translation_delta": 0,
+
+          # Pooling type following the wide convolutional layer.
+          "pooling_type": "max",
+
+          # Dropout rate for the fully connected layers.
+          "dropout_rate": 0.0,
+      },
+  }
+  return config
+
+
+def local_global():
+  """Base config for a locally fully connected model with local/global views."""
+  config = parent_configs.base()
+
+  # Override the model features to be local_view and global_view time series.
+  config["inputs"]["features"] = {
+      "local_view": {
+          "length": 201,
+          "is_time_series": True,
+      },
+      "global_view": {
+          "length": 2001,
+          "name_in_proto": "light_curve",
+          "is_time_series": True,
+          "data_source": "",
+      },
+  }
+
+  # Add configurations for the fully-connected layers of time series features.
+  config["hparams"]["time_series_hidden"] = {
+      "local_view": {
+          "num_local_layers": 0,
+          "local_layer_size": 128,
+          "translation_delta": 0,  # For wide convolution.
+          "pooling_type": "max",  # For wide convolution.
+          "dropout_rate": 0.0,
+      },
+      "global_view": {
+          "num_local_layers": 0,
+          "local_layer_size": 128,
+          "translation_delta": 0,  # For wide convolution.
+          "pooling_type": "max",  # For wide convolution.
+          "dropout_rate": 0.0,
+      },
+  }
+  return config

research/astronet/astronet/astro_model/BUILD

+package(default_visibility = ["//visibility:public"])
+
+licenses(["notice"])  # Apache 2.0
+
+py_library(
+    name = "configurations",
+    srcs = ["configurations.py"],
+    srcs_version = "PY2AND3",
+)
+
+py_library(
+    name = "astro_model",
+    srcs = [
+        "astro_model.py",
+    ],
+    srcs_version = "PY2AND3",
+)
+
+py_test(
+    name = "astro_model_test",
+    size = "small",
+    srcs = [
+        "astro_model_test.py",
+    ],
+    srcs_version = "PY2AND3",
+    deps = [
+        ":astro_model",
+        ":configurations",
+        "//astronet/ops:input_ops",
+        "//astronet/ops:testing",
+        "//astronet/util:configdict",
+    ],
+)

research/astronet/astronet/astro_model/__init__.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+

research/astronet/astronet/astro_model/astro_model.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""A TensorFlow model for identifying exoplanets in astrophysical light curves.
+
+AstroModel is a concrete base class for models that identify exoplanets in
+astrophysical light curves. This class implements a simple linear model that can
+be extended by subclasses.
+
+The general framework for AstroModel and its subclasses is as follows:
+
+  * Model inputs:
+     - Zero or more time_series_features (e.g. astrophysical light curves)
+     - Zero or more aux_features (e.g. orbital period, transit duration)
+
+  * Labels:
+     - An integer feature with 2 or more values (eg. 0 = Not Planet, 1 = Planet)
+
+  * Model outputs:
+     - The predicted probabilities for each label
+
+  * Architecture:
+
+                         predictions
+                              ^
+                              |
+                           logits
+                              ^
+                              |
+                   (pre_logits_hidden_layers)
+                              ^
+                              |
+                       pre_logits_concat
+                              ^
+                              |
+                        (concatenate)
+                ^                           ^
+                |                           |
+     (time_series_hidden_layers)    (aux_hidden_layers)
+                ^                           ^
+                |                           |
+       time_series_features           aux_features
+
+
+Subclasses will typically override the build_time_series_hidden_layers()
+and/or build_aux_hidden_layers() functions. For example, a subclass could
+override build_time_series_hidden_layers() to apply convolutional layers to the
+time series features. In this class, those functions are simple concatenations
+of the input features.
+"""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import operator
+
+import tensorflow as tf
+
+
+class AstroModel(object):
+  """A TensorFlow model for classifying astrophysical light curves."""
+
+  def __init__(self, features, labels, hparams, mode):
+    """Basic setup. The actual TensorFlow graph is constructed in build().
+
+    Args:
+      features: A dictionary containing "time_series_features" and
+          "aux_features", each of which is a dictionary of named input Tensors.
+          All features have dtype float32 and shape [batch_size, length].
+      labels: An int64 Tensor with shape [batch_size]. May be None if mode is
+          tf.estimator.ModeKeys.PREDICT.
+      hparams: A ConfigDict of hyperparameters for building the model.
+      mode: A tf.estimator.ModeKeys to specify whether the graph should be built
+          for training, evaluation or prediction.
+
+    Raises:
+      ValueError: If mode is invalid.
+    """
+    valid_modes = [
+        tf.estimator.ModeKeys.TRAIN, tf.estimator.ModeKeys.EVAL,
+        tf.estimator.ModeKeys.PREDICT
+    ]
+    if mode not in valid_modes:
+      raise ValueError("Expected mode in %s. Got: %s" % (valid_modes, mode))
+
+    self.hparams = hparams
+    self.mode = mode
+
+    # A dictionary of input Tensors. Values have dtype float32 and shape
+    # [batch_size, length].
+    self.time_series_features = features.get("time_series_features", {})
+
+    # A dictionary of input Tensors. Values have dtype float32 and shape
+    # [batch_size, length].
+    self.aux_features = features.get("aux_features", {})
+
+    # An int32 Tensor with shape [batch_size]. May be None if mode is
+    # tf.estimator.ModeKeys.PREDICT.
+    self.labels = labels
+
+    # Optional Tensor; the weights corresponding to self.labels.
+    self.weights = features.get("weights")
+
+    # A Python boolean or a scalar boolean Tensor. Indicates whether the model
+    # is in training mode for the purpose of graph ops, such as dropout. (Since
+    # this might be a Tensor, its value is defined in build()).
+    self.is_training = None
+
+    # Global step Tensor.
+    self.global_step = None
+
+    # A dictionary of float32 Tensors with shape [batch_size, layer_size]; the
+    # outputs of the time series hidden layers.
+    self.time_series_hidden_layers = {}
+
+    # A dictionary of float32 Tensors with shape [batch_size, layer_size]; the
+    # outputs of the auxiliary hidden layers.
+    self.aux_hidden_layers = {}
+
+    # A float32 Tensor with shape [batch_size, layer_size]; the concatenation of
+    # outputs from the hidden layers.
+    self.pre_logits_concat = None
+
+    # A float32 Tensor with shape [batch_size, output_dim].
+    self.logits = None
+
+    # A float32 Tensor with shape [batch_size, output_dim].
+    self.predictions = None
+
+    # A float32 Tensor with shape [batch_size]; the cross-entropy losses for the
+    # current batch.
+    self.batch_losses = None
+
+    # Scalar Tensor; the total loss for the trainer to optimize.
+    self.total_loss = None
+
+  def build_time_series_hidden_layers(self):
+    """Builds hidden layers for the time series features.
+
+    Inputs:
+      self.time_series_features
+
+    Outputs:
+      self.time_series_hidden_layers
+    """
+    # No hidden layers.
+    self.time_series_hidden_layers = self.time_series_features
+
+  def build_aux_hidden_layers(self):
+    """Builds hidden layers for the auxiliary features.
+
+    Inputs:
+      self.aux_features
+
+    Outputs:
+      self.aux_hidden_layers
+    """
+    # No hidden layers.
+    self.aux_hidden_layers = self.aux_features
+
+  def build_logits(self):
+    """Builds the model logits.
+
+    Inputs:
+      self.aux_hidden_layers
+      self.time_series_hidden_layers
+
+    Outputs:
+      self.pre_logits_concat
+      self.logits
+
+    Raises:
+      ValueError: If self.time_series_hidden_layers and self.aux_hidden_layers
+          are both empty.
+    """
+    # Sort the hidden layers by name because the order of dictionary items is
+    # nondeterministic between invocations of Python.
+    time_series_hidden_layers = sorted(
+        self.time_series_hidden_layers.items(), key=operator.itemgetter(0))
+    aux_hidden_layers = sorted(
+        self.aux_hidden_layers.items(), key=operator.itemgetter(0))
+
+    hidden_layers = time_series_hidden_layers + aux_hidden_layers
+    if not hidden_layers:
+      raise ValueError("At least one time series hidden layer or auxiliary "
+                       "hidden layer is required.")
+
+    # Concatenate the hidden layers.
+    if len(hidden_layers) == 1:
+      pre_logits_concat = hidden_layers[0][1]
+    else:
+      pre_logits_concat = tf.concat(
+          [layer[1] for layer in hidden_layers],
+          axis=1,
+          name="pre_logits_concat")
+
+    net = pre_logits_concat
+    with tf.variable_scope("pre_logits_hidden"):
+      for i in range(self.hparams.num_pre_logits_hidden_layers):
+        net = tf.layers.dense(
+            inputs=net,
+            units=self.hparams.pre_logits_hidden_layer_size,
+            activation=tf.nn.relu,
+            name="fully_connected_%s" % (i + 1))
+
+        if self.hparams.pre_logits_dropout_rate > 0:
+          net = tf.layers.dropout(
+              net,
+              self.hparams.pre_logits_dropout_rate,
+              training=self.is_training)
+
+      # Identify the final pre-logits hidden layer as "pre_logits_hidden/final".
+      tf.identity(net, "final")
+
+    logits = tf.layers.dense(
+        inputs=net, units=self.hparams.output_dim, name="logits")
+
+    self.pre_logits_concat = pre_logits_concat
+    self.logits = logits
+
+  def build_predictions(self):
+    """Builds the output predictions and losses.
+
+    Inputs:
+      self.logits
+
+    Outputs:
+      self.predictions
+    """
+    # Use sigmoid activation function for binary classification, or softmax for
+    # multi-class classification.
+    prediction_fn = (
+        tf.sigmoid if self.hparams.output_dim == 1 else tf.nn.softmax)
+    predictions = prediction_fn(self.logits, name="predictions")
+
+    self.predictions = predictions
+
+  def build_losses(self):
+    """Builds the training losses.
+
+    Inputs:
+      self.logits
+      self.labels
+
+    Outputs:
+      self.batch_losses
+      self.total_loss
+    """
+
+    if self.hparams.output_dim == 1:
+      # Binary classification.
+      batch_losses = tf.nn.sigmoid_cross_entropy_with_logits(
+          labels=tf.to_float(self.labels), logits=tf.squeeze(self.logits, [1]))
+    else:
+      # Multi-class classification.
+      batch_losses = tf.nn.sparse_softmax_cross_entropy_with_logits(
+          labels=self.labels, logits=self.logits)
+
+    # Compute the weighted mean cross entropy loss and add it to the LOSSES
+    # collection.
+    weights = self.weights if self.weights is not None else 1.0
+    tf.losses.compute_weighted_loss(
+        losses=batch_losses,
+        weights=weights,
+        reduction=tf.losses.Reduction.MEAN)
+
+    # Compute the total loss, including any other losses added to the LOSSES
+    # collection (e.g. regularization losses).
+    total_loss = tf.losses.get_total_loss()
+
+    self.batch_losses = batch_losses
+    self.total_loss = total_loss
+
+  def build(self):
+    """Creates all ops for training, evaluation or inference."""
+    self.global_step = tf.train.get_or_create_global_step()
+
+    if self.mode == tf.estimator.ModeKeys.TRAIN:
+      # This is implemented as a placeholder Tensor, rather than a constant, to
+      # allow its value to be feedable during training (e.g. to disable dropout
+      # when performing in-process validation set evaluation).
+      self.is_training = tf.placeholder_with_default(True, [], "is_training")
+    else:
+      self.is_training = False
+
+    self.build_time_series_hidden_layers()
+    self.build_aux_hidden_layers()
+    self.build_logits()
+    self.build_predictions()
+
+    if self.mode in [tf.estimator.ModeKeys.TRAIN, tf.estimator.ModeKeys.EVAL]:
+      self.build_losses()

research/astronet/astronet/astro_model/astro_model_test.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Tests for astro_model.AstroModel."""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import numpy as np
+import tensorflow as tf
+
+from astronet.astro_model import astro_model
+from astronet.astro_model import configurations
+from astronet.ops import input_ops
+from astronet.ops import testing
+from astronet.util import configdict
+
+
+class AstroModelTest(tf.test.TestCase):
+
+  def assertShapeEquals(self, shape, tensor_or_array):
+    """Asserts that a Tensor or Numpy array has the expected shape.
+
+    Args:
+      shape: Numpy array or anything that can be converted to one.
+      tensor_or_array: tf.Tensor, tf.Variable, Numpy array or anything that can
+          be converted to one.
+    """
+    if isinstance(tensor_or_array, (np.ndarray, np.generic)):
+      self.assertAllEqual(shape, tensor_or_array.shape)
+    elif isinstance(tensor_or_array, (tf.Tensor, tf.Variable)):
+      self.assertAllEqual(shape, tensor_or_array.shape.as_list())
+    else:
+      raise TypeError("tensor_or_array must be a Tensor or Numpy ndarray")
+
+  def testInvalidModeRaisesError(self):
+    # Build config.
+    config = configdict.ConfigDict(configurations.base())
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    with self.assertRaises(ValueError):
+      _ = astro_model.AstroModel(features, labels, config.hparams, "training")
+
+  def testZeroFeaturesRaisesError(self):
+    # Build config.
+    config = configurations.base()
+    config["inputs"]["features"] = {}
+    config = configdict.ConfigDict(config)
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    with self.assertRaises(ValueError):
+      # Raises ValueError because at least one feature is required.
+      model.build()
+
+  def testOneTimeSeriesFeature(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        }
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate hidden layers.
+    self.assertItemsEqual(["time_feature_1"],
+                          model.time_series_hidden_layers.keys())
+    self.assertIs(model.time_series_features["time_feature_1"],
+                  model.time_series_hidden_layers["time_feature_1"])
+    self.assertEqual(len(model.aux_hidden_layers), 0)
+    self.assertIs(model.time_series_features["time_feature_1"],
+                  model.pre_logits_concat)
+
+  def testOneAuxFeature(self):
+    # Build config.
+    feature_spec = {
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        }
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate hidden layers.
+    self.assertEqual(len(model.time_series_hidden_layers), 0)
+    self.assertItemsEqual(["aux_feature_1"], model.aux_hidden_layers.keys())
+    self.assertIs(model.aux_features["aux_feature_1"],
+                  model.aux_hidden_layers["aux_feature_1"])
+    self.assertIs(model.aux_features["aux_feature_1"], model.pre_logits_concat)
+
+  def testTwoTimeSeriesFeatures(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "time_feature_2": {
+            "length": 10,
+            "is_time_series": True,
+        }
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate hidden layers.
+    self.assertItemsEqual(["time_feature_1", "time_feature_2"],
+                          model.time_series_hidden_layers.keys())
+    self.assertIs(model.time_series_features["time_feature_1"],
+                  model.time_series_hidden_layers["time_feature_1"])
+    self.assertIs(model.time_series_features["time_feature_2"],
+                  model.time_series_hidden_layers["time_feature_2"])
+    self.assertEqual(len(model.aux_hidden_layers), 0)
+    self.assertShapeEquals((None, 20), model.pre_logits_concat)
+
+  def testOneTimeSeriesOneAuxFeature(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate hidden layers.
+    self.assertItemsEqual(["time_feature_1"],
+                          model.time_series_hidden_layers.keys())
+    self.assertIs(model.time_series_features["time_feature_1"],
+                  model.time_series_hidden_layers["time_feature_1"])
+    self.assertItemsEqual(["aux_feature_1"], model.aux_hidden_layers.keys())
+    self.assertIs(model.aux_features["aux_feature_1"],
+                  model.aux_hidden_layers["aux_feature_1"])
+    self.assertShapeEquals((None, 11), model.pre_logits_concat)
+
+  def testOneTimeSeriesTwoAuxFeatures(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+        "aux_feature_2": {
+            "length": 2,
+            "is_time_series": False,
+        },
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate hidden layers.
+    self.assertItemsEqual(["time_feature_1"],
+                          model.time_series_hidden_layers.keys())
+    self.assertIs(model.time_series_features["time_feature_1"],
+                  model.time_series_hidden_layers["time_feature_1"])
+    self.assertItemsEqual(["aux_feature_1", "aux_feature_2"],
+                          model.aux_hidden_layers.keys())
+    self.assertIs(model.aux_features["aux_feature_1"],
+                  model.aux_hidden_layers["aux_feature_1"])
+    self.assertIs(model.aux_features["aux_feature_2"],
+                  model.aux_hidden_layers["aux_feature_2"])
+    self.assertShapeEquals((None, 13), model.pre_logits_concat)
+
+  def testZeroHiddenLayers(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "time_feature_2": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+    config.hparams.output_dim = 1
+    config.hparams.num_pre_logits_hidden_layers = 0
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate Tensor shapes.
+    self.assertShapeEquals((None, 21), model.pre_logits_concat)
+    logits_w = testing.get_variable_by_name("logits/kernel")
+    self.assertShapeEquals((21, 1), logits_w)
+
+  def testOneHiddenLayer(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "time_feature_2": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+    config.hparams.output_dim = 1
+    config.hparams.num_pre_logits_hidden_layers = 1
+    config.hparams.pre_logits_hidden_layer_size = 5
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate Tensor shapes.
+    self.assertShapeEquals((None, 21), model.pre_logits_concat)
+    fc1 = testing.get_variable_by_name(
+        "pre_logits_hidden/fully_connected_1/kernel")
+    self.assertShapeEquals((21, 5), fc1)
+    logits_w = testing.get_variable_by_name("logits/kernel")
+    self.assertShapeEquals((5, 1), logits_w)
+
+  def testTwoHiddenLayers(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "time_feature_2": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+    config.hparams.output_dim = 1
+    config.hparams.num_pre_logits_hidden_layers = 2
+    config.hparams.pre_logits_hidden_layer_size = 5
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate Tensor shapes.
+    self.assertShapeEquals((None, 21), model.pre_logits_concat)
+    fc1 = testing.get_variable_by_name(
+        "pre_logits_hidden/fully_connected_1/kernel")
+    self.assertShapeEquals((21, 5), fc1)
+    fc2 = testing.get_variable_by_name(
+        "pre_logits_hidden/fully_connected_2/kernel")
+    self.assertShapeEquals((5, 5), fc2)
+    logits_w = testing.get_variable_by_name("logits/kernel")
+    self.assertShapeEquals((5, 1), logits_w)
+
+  def testBinaryClassification(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "time_feature_2": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+    config.hparams.output_dim = 1
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate Tensor shapes.
+    self.assertShapeEquals((None, 1), model.logits)
+    self.assertShapeEquals((None, 1), model.predictions)
+    self.assertShapeEquals((None,), model.batch_losses)
+    self.assertShapeEquals((), model.total_loss)
+
+    # Execute the TensorFlow graph.
+    scaffold = tf.train.Scaffold()
+    scaffold.finalize()
+    with self.test_session() as sess:
+      sess.run([scaffold.init_op, scaffold.local_init_op])
+      step = sess.run(model.global_step)
+      self.assertEqual(0, step)
+
+      # Fetch total loss.
+      features = testing.fake_features(feature_spec, batch_size=16)
+      labels = testing.fake_labels(config.hparams.output_dim, batch_size=16)
+      feed_dict = input_ops.prepare_feed_dict(model, features, labels)
+      total_loss = sess.run(model.total_loss, feed_dict=feed_dict)
+      self.assertShapeEquals((), total_loss)
+
+  def testMultiLabelClassification(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "time_feature_2": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+    config.hparams.output_dim = 3
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate Tensor shapes.
+    self.assertShapeEquals((None, 3), model.logits)
+    self.assertShapeEquals((None, 3), model.predictions)
+    self.assertShapeEquals((None,), model.batch_losses)
+    self.assertShapeEquals((), model.total_loss)
+
+    # Execute the TensorFlow graph.
+    scaffold = tf.train.Scaffold()
+    scaffold.finalize()
+    with self.test_session() as sess:
+      sess.run([scaffold.init_op, scaffold.local_init_op])
+      step = sess.run(model.global_step)
+      self.assertEqual(0, step)
+
+      # Fetch total loss.
+      features = testing.fake_features(feature_spec, batch_size=16)
+      labels = testing.fake_labels(config.hparams.output_dim, batch_size=16)
+      feed_dict = input_ops.prepare_feed_dict(model, features, labels)
+      total_loss = sess.run(model.total_loss, feed_dict=feed_dict)
+      self.assertShapeEquals((), total_loss)
+
+  def testEvalMode(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "time_feature_2": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+    config.hparams.output_dim = 1
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    labels = input_ops.build_labels_placeholder()
+    model = astro_model.AstroModel(features, labels, config.hparams,
+                                   tf.estimator.ModeKeys.TRAIN)
+    model.build()
+
+    # Validate Tensor shapes.
+    self.assertShapeEquals((None, 21), model.pre_logits_concat)
+    self.assertShapeEquals((None, 1), model.logits)
+    self.assertShapeEquals((None, 1), model.predictions)
+    self.assertShapeEquals((None,), model.batch_losses)
+    self.assertShapeEquals((), model.total_loss)
+
+    # Execute the TensorFlow graph.
+    scaffold = tf.train.Scaffold()
+    scaffold.finalize()
+    with self.test_session() as sess:
+      sess.run([scaffold.init_op, scaffold.local_init_op])
+      step = sess.run(model.global_step)
+      self.assertEqual(0, step)
+
+      # Fetch total loss.
+      features = testing.fake_features(feature_spec, batch_size=16)
+      labels = testing.fake_labels(config.hparams.output_dim, batch_size=16)
+      feed_dict = input_ops.prepare_feed_dict(model, features, labels)
+      total_loss = sess.run(model.total_loss, feed_dict=feed_dict)
+      self.assertShapeEquals((), total_loss)
+
+  def testPredictMode(self):
+    # Build config.
+    feature_spec = {
+        "time_feature_1": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "time_feature_2": {
+            "length": 10,
+            "is_time_series": True,
+        },
+        "aux_feature_1": {
+            "length": 1,
+            "is_time_series": False,
+        },
+    }
+    config = configurations.base()
+    config["inputs"]["features"] = feature_spec
+    config = configdict.ConfigDict(config)
+    config.hparams.output_dim = 1
+
+    # Build model.
+    features = input_ops.build_feature_placeholders(config.inputs.features)
+    model = astro_model.AstroModel(features, None, config.hparams,
+                                   tf.estimator.ModeKeys.PREDICT)
+    model.build()
+
+    # Validate Tensor shapes.
+    self.assertIsNone(model.labels)
+    self.assertShapeEquals((None, 21), model.pre_logits_concat)
+    self.assertShapeEquals((None, 1), model.logits)
+    self.assertShapeEquals((None, 1), model.predictions)
+    self.assertIsNone(model.batch_losses)
+    self.assertIsNone(model.total_loss)
+
+    # Execute the TensorFlow graph.
+    scaffold = tf.train.Scaffold()
+    scaffold.finalize()
+    with self.test_session() as sess:
+      sess.run([scaffold.init_op, scaffold.local_init_op])
+      step = sess.run(model.global_step)
+      self.assertEqual(0, step)
+
+      # Fetch predictions.
+      features = testing.fake_features(feature_spec, batch_size=16)
+      feed_dict = input_ops.prepare_feed_dict(model, features)
+      predictions = sess.run(model.predictions, feed_dict=feed_dict)
+      self.assertShapeEquals((16, 1), predictions)
+
+
+if __name__ == "__main__":
+  tf.test.main()

research/astronet/astronet/astro_model/configurations.py

+# Copyright 2018 The TensorFlow Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Configurations for model building, training and evaluation.
+
+The default base configuration has one "global_view" time series feature per
+input example. Additional time series features and auxiliary features can be
+added.
+"""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+
+def base():
+  """Returns the base config for model building, training and evaluation."""
+  return {
+      # Configuration for reading input features and labels.
+      "inputs": {
+          # Feature specifications.
+          "features": {
+              "global_view": {
+                  "length": 2001,
+                  "is_time_series": True,
+              },
+          },
+
+          # Name of the feature containing training labels.
+          "label_feature": "av_training_set",
+
+          # Label string to integer id.
+          "label_map": {
+              "PC": 1,  # Planet Candidate.
+              "AFP": 0,  # Astrophysical False Positive.
+              "NTP": 0,  # Non-Transiting Phenomenon.
+          },
+      },
+      # Hyperparameters for building and training the model.
+      "hparams": {
+          # Number of output dimensions (predictions) for the classification
+          # task. If >= 2 then a softmax output layer is used. If equal to 1
+          # then a sigmoid output layer is used.
+          "output_dim": 1,
+
+          # Fully connected layers before the logits layer.
+          "num_pre_logits_hidden_layers": 0,
+          "pre_logits_hidden_layer_size": 0,
+          "pre_logits_dropout_rate": 0.0,
+
+          # Number of examples per training batch.
+          "batch_size": 64,
+
+          # Learning rate parameters.
+          "learning_rate": 1e-5,
+          "learning_rate_decay_steps": 0,
+          "learning_rate_decay_factor": 0,
+          "learning_rate_decay_staircase": True,
+
+          # Optimizer for training the model.
+          "optimizer": "adam",
+
+          # If not None, gradient norms will be clipped to this value.
+          "clip_gradient_norm": None,
+      }
+  }