@@ -9,7 +9,7 @@ We use Gradient Boosted Trees algorithm to distinguish the two classes.
...
@@ -9,7 +9,7 @@ We use Gradient Boosted Trees algorithm to distinguish the two classes.
The code sample uses the high level `tf.estimator.Estimator` and `tf.data.Dataset`. These APIs are great for fast iteration and quickly adapting models to your own datasets without major code overhauls. It allows you to move from single-worker training to distributed training, and makes it easy to export model binaries for prediction. Here, for further simplicity and faster execution, we use a utility function `tf.contrib.estimator.boosted_trees_classifier_train_in_memory`. This utility function is especially effective when the input is provided as in-memory data sets like numpy arrays.
The code sample uses the high level `tf.estimator.Estimator` and `tf.data.Dataset`. These APIs are great for fast iteration and quickly adapting models to your own datasets without major code overhauls. It allows you to move from single-worker training to distributed training, and makes it easy to export model binaries for prediction. Here, for further simplicity and faster execution, we use a utility function `tf.contrib.estimator.boosted_trees_classifier_train_in_memory`. This utility function is especially effective when the input is provided as in-memory data sets like numpy arrays.
An input function for the `Estimator` typically uses `tf.data.Dataset` API, which can handle various data control like streaming, batching, transform and shuffling. However `boosted_trees_classifier_train_in_memory()` utility function requires that the entire data is provided as a single batch (i.e. without using `batch()` API). Thus in this practice, simply `Dataset.from_tensors()` is used to convert numpy arrays into structured tensors, and `Dataset.zip()` is used to put features and label together.
An input function for the `Estimator` typically uses `tf.data.Dataset` API, which can handle various data control like streaming, batching, transform and shuffling. However `boosted_trees_classifier_train_in_memory()` utility function requires that the entire data is provided as a single batch (i.e. without using `batch()` API). Thus in this practice, simply `Dataset.from_tensors()` is used to convert numpy arrays into structured tensors, and `Dataset.zip()` is used to put features and label together.
For further references of `Dataset`, [Read more here](https://www.tensorflow.org/programmers_guide/datasets).
For further references of `Dataset`, [Read more here](https://www.tensorflow.org/guide/datasets).
## Running the code
## Running the code
First make sure you've [added the models folder to your Python path](/official/#running-the-models); otherwise you may encounter an error like `ImportError: No module named official.boosted_trees`.
First make sure you've [added the models folder to your Python path](/official/#running-the-models); otherwise you may encounter an error like `ImportError: No module named official.boosted_trees`.
...
@@ -53,13 +53,13 @@ tensorboard --logdir=/tmp/higgs_model # set logdir as --model_dir set during tr
...
@@ -53,13 +53,13 @@ tensorboard --logdir=/tmp/higgs_model # set logdir as --model_dir set during tr
```
```
## Inference with SavedModel
## Inference with SavedModel
You can export the model into Tensorflow [SavedModel](https://www.tensorflow.org/programmers_guide/saved_model) format by using the argument `--export_dir`:
You can export the model into Tensorflow [SavedModel](https://www.tensorflow.org/guide/saved_model) format by using the argument `--export_dir`:
After the model finishes training, use [`saved_model_cli`](https://www.tensorflow.org/programmers_guide/saved_model#cli_to_inspect_and_execute_savedmodel) to inspect and execute the SavedModel.
After the model finishes training, use [`saved_model_cli`](https://www.tensorflow.org/guide/saved_model#cli_to_inspect_and_execute_savedmodel) to inspect and execute the SavedModel.
Try the following commands to inspect the SavedModel:
Try the following commands to inspect the SavedModel:
You can export the model into Tensorflow [SavedModel](https://www.tensorflow.org/programmers_guide/saved_model) format by using the argument `--export_dir`:
You can export the model into Tensorflow [SavedModel](https://www.tensorflow.org/guide/saved_model) format by using the argument `--export_dir`:
The SavedModel will be saved in a timestamped directory under `/tmp/mnist_saved_model/` (e.g. `/tmp/mnist_saved_model/1513630966/`).
The SavedModel will be saved in a timestamped directory under `/tmp/mnist_saved_model/` (e.g. `/tmp/mnist_saved_model/1513630966/`).
**Getting predictions with SavedModel**
**Getting predictions with SavedModel**
Use [`saved_model_cli`](https://www.tensorflow.org/programmers_guide/saved_model#cli_to_inspect_and_execute_savedmodel) to inspect and execute the SavedModel.
Use [`saved_model_cli`](https://www.tensorflow.org/guide/saved_model#cli_to_inspect_and_execute_savedmodel) to inspect and execute the SavedModel.
```
```
saved_model_cli run --dir /tmp/mnist_saved_model/TIMESTAMP --tag_set serve --signature_def classify --inputs image=examples.npy
saved_model_cli run --dir /tmp/mnist_saved_model/TIMESTAMP --tag_set serve --signature_def classify --inputs image=examples.npy
demonstration purposes only, but will be optimized in the coming weeks.
demonstration purposes only, but will be optimized in the coming weeks.
## Export trained model
## Export trained model
To export the model as a Tensorflow [SavedModel](https://www.tensorflow.org/programmers_guide/saved_model) format, use the argument `--export_dir` when running `transformer_main.py`. A folder will be created in the directory with the name as the timestamp (e.g. $EXPORT_DIR/1526427396).
To export the model as a Tensorflow [SavedModel](https://www.tensorflow.org/guide/saved_model) format, use the argument `--export_dir` when running `transformer_main.py`. A folder will be created in the directory with the name as the timestamp (e.g. $EXPORT_DIR/1526427396).
@@ -10,7 +10,7 @@ For the purposes of this example code, the Census Income Data Set was chosen to
...
@@ -10,7 +10,7 @@ For the purposes of this example code, the Census Income Data Set was chosen to
The code sample in this directory uses the high level `tf.estimator.Estimator` API. This API is great for fast iteration and quickly adapting models to your own datasets without major code overhauls. It allows you to move from single-worker training to distributed training, and it makes it easy to export model binaries for prediction.
The code sample in this directory uses the high level `tf.estimator.Estimator` API. This API is great for fast iteration and quickly adapting models to your own datasets without major code overhauls. It allows you to move from single-worker training to distributed training, and it makes it easy to export model binaries for prediction.
The input function for the `Estimator` uses `tf.contrib.data.TextLineDataset`, which creates a `Dataset` object. The `Dataset` API makes it easy to apply transformations (map, batch, shuffle, etc.) to the data. [Read more here](https://www.tensorflow.org/programmers_guide/datasets).
The input function for the `Estimator` uses `tf.contrib.data.TextLineDataset`, which creates a `Dataset` object. The `Dataset` API makes it easy to apply transformations (map, batch, shuffle, etc.) to the data. [Read more here](https://www.tensorflow.org/guide/datasets).
The `Estimator` and `Dataset` APIs are both highly encouraged for fast development and efficient training.
The `Estimator` and `Dataset` APIs are both highly encouraged for fast development and efficient training.
You can export the model into Tensorflow [SavedModel](https://www.tensorflow.org/programmers_guide/saved_model) format by using the argument `--export_dir`:
You can export the model into Tensorflow [SavedModel](https://www.tensorflow.org/guide/saved_model) format by using the argument `--export_dir`:
After the model finishes training, use [`saved_model_cli`](https://www.tensorflow.org/programmers_guide/saved_model#cli_to_inspect_and_execute_savedmodel) to inspect and execute the SavedModel.
After the model finishes training, use [`saved_model_cli`](https://www.tensorflow.org/guide/saved_model#cli_to_inspect_and_execute_savedmodel) to inspect and execute the SavedModel.
Try the following commands to inspect the SavedModel:
Try the following commands to inspect the SavedModel:
"In this guide, we will train a neural network model to classify images of clothing, like sneakers and shirts. It's fine if you don't understand all the details, this is a fast-paced overview of a complete TensorFlow program with the details explained as we go.\n",
"In this guide, we will train a neural network model to classify images of clothing, like sneakers and shirts. It's fine if you don't understand all the details, this is a fast-paced overview of a complete TensorFlow program with the details explained as we go.\n",
"\n",
"\n",
"This guide uses [tf.keras](https://www.tensorflow.org/programmers_guide/keras), a high-level API to build and train models in TensorFlow."
"This guide uses [tf.keras](https://www.tensorflow.org/guide/keras), a high-level API to build and train models in TensorFlow."
"This notebook builds a model to predict the median price of homes in a Boston suburb during the mid-1970s. To do this, we'll provide the model with some data points about the suburb, such as the crime rate and the local property tax rate.\n",
"This notebook builds a model to predict the median price of homes in a Boston suburb during the mid-1970s. To do this, we'll provide the model with some data points about the suburb, such as the crime rate and the local property tax rate.\n",
"\n",
"\n",
"This example uses the `tf.keras` API, see [this guide](https://www.tensorflow.org/programmers_guide/keras) for details."
"This example uses the `tf.keras` API, see [this guide](https://www.tensorflow.org/guide/keras) for details."
"We'll use the [IMDB dataset](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/imdb) that contains the text of 50,000 movie reviews from the [Internet Movie Database](https://www.imdb.com/). These are split into 25,000 reviews for training and 25,000 reviews for testing. The training and testing sets are *balanced*, meaning they contain an equal number of positive and negative reviews. \n",
"We'll use the [IMDB dataset](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/imdb) that contains the text of 50,000 movie reviews from the [Internet Movie Database](https://www.imdb.com/). These are split into 25,000 reviews for training and 25,000 reviews for testing. The training and testing sets are *balanced*, meaning they contain an equal number of positive and negative reviews. \n",
"\n",
"\n",
"This notebook uses [tf.keras](https://www.tensorflow.org/programmers_guide/keras), a high-level API to build and train models in TensorFlow."
"This notebook uses [tf.keras](https://www.tensorflow.org/guide/keras), a high-level API to build and train models in TensorFlow."
"There are many [TensorFlow APIs](https://www.tensorflow.org/api_docs/python/) available, but start with these high-level TensorFlow concepts:\n",
"There are many [TensorFlow APIs](https://www.tensorflow.org/api_docs/python/) available, but start with these high-level TensorFlow concepts:\n",
"\n",
"\n",
"* Enable an [eager execution](https://www.tensorflow.org/programmers_guide/eager) development environment,\n",
"* Enable an [eager execution](https://www.tensorflow.org/guide/eager) development environment,\n",
"* Import data with the [Datasets API](https://www.tensorflow.org/programmers_guide/datasets),\n",
"* Import data with the [Datasets API](https://www.tensorflow.org/guide/datasets),\n",
"* Build models and layers with TensorFlow's [Keras API](https://keras.io/getting-started/sequential-model-guide/).\n",
"* Build models and layers with TensorFlow's [Keras API](https://keras.io/getting-started/sequential-model-guide/).\n",
"\n",
"\n",
"This tutorial is structured like many TensorFlow programs:\n",
"This tutorial is structured like many TensorFlow programs:\n",
...
@@ -155,9 +155,9 @@
...
@@ -155,9 +155,9 @@
"source": [
"source": [
"### Configure imports and eager execution\n",
"### Configure imports and eager execution\n",
"\n",
"\n",
"Import the required Python modules—including TensorFlow—and enable eager execution for this program. Eager execution makes TensorFlow evaluate operations immediately, returning concrete values instead of creating a [computational graph](https://www.tensorflow.org/programmers_guide/graphs) that is executed later. If you are used to a REPL or the `python` interactive console, this feels familiar.\n",
"Import the required Python modules—including TensorFlow—and enable eager execution for this program. Eager execution makes TensorFlow evaluate operations immediately, returning concrete values instead of creating a [computational graph](https://www.tensorflow.org/guide/graphs) that is executed later. If you are used to a REPL or the `python` interactive console, this feels familiar.\n",
"\n",
"\n",
"Once eager execution is enabled, it *cannot* be disabled within the same program. See the [eager execution guide](https://www.tensorflow.org/programmers_guide/eager) for more details."
"Once eager execution is enabled, it *cannot* be disabled within the same program. See the [eager execution guide](https://www.tensorflow.org/guide/eager) for more details."
]
]
},
},
{
{
...
@@ -349,7 +349,7 @@
...
@@ -349,7 +349,7 @@
"source": [
"source": [
"### Create a `tf.data.Dataset`\n",
"### Create a `tf.data.Dataset`\n",
"\n",
"\n",
"TensorFlow's [Dataset API](https://www.tensorflow.org/programmers_guide/datasets) handles many common cases for loading data into a model. This is a high-level API for reading data and transforming it into a form used for training. See the [Datasets Quick Start guide](https://www.tensorflow.org/get_started/datasets_quickstart) for more information.\n",
"TensorFlow's [Dataset API](https://www.tensorflow.org/guide/datasets) handles many common cases for loading data into a model. This is a high-level API for reading data and transforming it into a form used for training. See the [Datasets Quick Start guide](https://www.tensorflow.org/get_started/datasets_quickstart) for more information.\n",
"\n",
"\n",
"\n",
"\n",
"Since the dataset is a CSV-formatted text file, use the the [make_csv_dataset](https://www.tensorflow.org/api_docs/python/tf/contrib/data/make_csv_dataset) function to parse the data into a suitable format. Since this function generates data for training models, the default behavior is to shuffle the data (`shuffle=True, shuffle_buffer_size=10000`), and repeat the dataset forever (`num_epochs=None`). We also set the [batch_size](https://developers.google.com/machine-learning/glossary/#batch_size) parameter."
"Since the dataset is a CSV-formatted text file, use the the [make_csv_dataset](https://www.tensorflow.org/api_docs/python/tf/contrib/data/make_csv_dataset) function to parse the data into a suitable format. Since this function generates data for training models, the default behavior is to shuffle the data (`shuffle=True, shuffle_buffer_size=10000`), and repeat the dataset forever (`num_epochs=None`). We also set the [batch_size](https://developers.google.com/machine-learning/glossary/#batch_size) parameter."
...
@@ -713,7 +713,7 @@
...
@@ -713,7 +713,7 @@
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"Use the [tf.GradientTape](https://www.tensorflow.org/api_docs/python/tf/GradientTape) context to calculate the *[gradients](https://developers.google.com/machine-learning/crash-course/glossary#gradient)* used to optimize our model. For more examples of this, see the [eager execution guide](https://www.tensorflow.org/programmers_guide/eager)."
"Use the [tf.GradientTape](https://www.tensorflow.org/api_docs/python/tf/GradientTape) context to calculate the *[gradients](https://developers.google.com/machine-learning/crash-course/glossary#gradient)* used to optimize our model. For more examples of this, see the [eager execution guide](https://www.tensorflow.org/guide/eager)."
]
]
},
},
{
{
...
@@ -894,7 +894,7 @@
...
@@ -894,7 +894,7 @@
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"While it's helpful to print out the model's training progress, it's often *more* helpful to see this progress. [TensorBoard](https://www.tensorflow.org/programmers_guide/summaries_and_tensorboard) is a nice visualization tool that is packaged with TensorFlow, but we can create basic charts using the `matplotlib` module.\n",
"While it's helpful to print out the model's training progress, it's often *more* helpful to see this progress. [TensorBoard](https://www.tensorflow.org/guide/summaries_and_tensorboard) is a nice visualization tool that is packaged with TensorFlow, but we can create basic charts using the `matplotlib` module.\n",
"\n",
"\n",
"Interpreting these charts takes some experience, but you really want to see the *loss* go down and the *accuracy* go up."
"Interpreting these charts takes some experience, but you really want to see the *loss* go down and the *accuracy* go up."
]
]
...
@@ -1123,7 +1123,7 @@
...
@@ -1123,7 +1123,7 @@
"source": [
"source": [
"These predictions look good!\n",
"These predictions look good!\n",
"\n",
"\n",
"To dig deeper into machine learning models, take a look at the TensorFlow [Programmer's Guide](https://www.tensorflow.org/programmers_guide/) and check out the [community](https://www.tensorflow.org/community/)."
"To dig deeper into machine learning models, take a look at the [TensorFlow Guide](https://www.tensorflow.org/guide/) and check out the [community](https://www.tensorflow.org/community/)."
"This notebook is based on the book [Deep Learning with Python](https://manning.com/books/deep-learning-with-python), which is a great way to continue learning more about Deep Learning and the Keras API—especially if you enjoy this style of exploring machine learning concepts with code examples. \n",
"This notebook is based on the book [Deep Learning with Python](https://manning.com/books/deep-learning-with-python), which is a great way to continue learning more about Deep Learning and the Keras API—especially if you enjoy this style of exploring machine learning concepts with code examples. \n",
"\n",
"\n",
"As always, the code in this example will use the `tf.keras` API, which you can learn more about in the TensorFlow [Keras guide](https://www.tensorflow.org/programmers_guide/keras).\n",
"As always, the code in this example will use the `tf.keras` API, which you can learn more about in the TensorFlow [Keras guide](https://www.tensorflow.org/guide/keras).\n",
"\n",
"\n",
"In both of the previous examples—classifying movie reviews, and predicting housing prices—we saw that the accuracy of our model on the validation data would peak after training for a number of epochs, and would then start decreasing. \n",
"In both of the previous examples—classifying movie reviews, and predicting housing prices—we saw that the accuracy of our model on the validation data would peak after training for a number of epochs, and would then start decreasing. \n",
"There are different ways to save TensorFlow models—depending on the API you're using. This guide uses [tf.keras](https://www.tensorflow.org/programmers_guide/keras), a high-level API to build and train models in TensorFlow. For other approaches, see the TensorFlow [Save and Restore](https://www.tensorflow.org/programmers_guide/saved_model) guide or [Saving in eager](https://www.tensorflow.org/programmers_guide/eager#object_based_saving).\n"
"There are different ways to save TensorFlow models—depending on the API you're using. This guide uses [tf.keras](https://www.tensorflow.org/guide/keras), a high-level API to build and train models in TensorFlow. For other approaches, see the TensorFlow [Save and Restore](https://www.tensorflow.org/guide/saved_model) guide or [Saving in eager](https://www.tensorflow.org/guide/eager#object_based_saving).\n"
]
]
},
},
{
{
...
@@ -502,7 +502,7 @@
...
@@ -502,7 +502,7 @@
},
},
"cell_type": "markdown",
"cell_type": "markdown",
"source": [
"source": [
"The above code stores the weights to a collection of [checkpoint](https://www.tensorflow.org/programmers_guide/saved_model#save_and_restore_variables)-formatted files that contains only the trained weights in a binary format. Checkpoints contain:\n",
"The above code stores the weights to a collection of [checkpoint](https://www.tensorflow.org/guide/saved_model#save_and_restore_variables)-formatted files that contains only the trained weights in a binary format. Checkpoints contain:\n",
"* One or more shards that contain your model's weights. \n",
"* One or more shards that contain your model's weights. \n",
"* An index file that indicates which weights are stored in a which shard. \n",
"* An index file that indicates which weights are stored in a which shard. \n",
"\n",
"\n",
...
@@ -654,11 +654,11 @@
...
@@ -654,11 +654,11 @@
"\n",
"\n",
"That was a quick guide to saving and loading in with `tf.keras`.\n",
"That was a quick guide to saving and loading in with `tf.keras`.\n",
"\n",
"\n",
"* The [tf.keras guide](https://www.tensorflow.org/programmers_guide/keras) shows more about saving and loading models with `tf.keras`.\n",
"* The [tf.keras guide](https://www.tensorflow.org/guide/keras) shows more about saving and loading models with `tf.keras`.\n",
"\n",
"\n",
"* See [Saving in eager](https://www.tensorflow.org/programmers_guide/eager#object_based_saving) for saving during eager execution.\n",
"* See [Saving in eager](https://www.tensorflow.org/guide/eager#object_based_saving) for saving during eager execution.\n",
"\n",
"\n",
"* The [Save and Restore](https://www.tensorflow.org/programmers_guide/saved_model) guide has low-level details about TensorFlow saving."
"* The [Save and Restore](https://www.tensorflow.org/guide/saved_model) guide has low-level details about TensorFlow saving."